We're running Sophos Protection for Linux Server (SPL) with MTR and InterceptX. We're running it across many Ubuntu 20.04 LTS servers (OS and apps are fully updated). We're also running Veeam Backup & Replication (VBR) with Veeam Agent for Linux (VAL) installed on all Ubuntu servers.
On our Ubuntu Linux servers, our Veeam backup software creates temporary filesystems similar to the following:
and mounts it as /dev/veeamimage0.
The hex string between the brackets is different for each backup session. At the end of the backup session, the temporary filesystem is emptied and unmounted. However, Sophos SPL appears to be doing something to "lock" the above filesystem and is preventing it from being unmounted at the end of the backup session. This is causing the backup session to fail. This is a big problem for us, and is impacting many production servers.
Our question is this: Is SPL doing something to lock down certain filesystems to prevent them from being unmounted, and if so, how can we exempt a specific filesystem so that Sophos does not lock it down?
This does NOT seem to be related to AV exceptions, as the above filesystem is already exempted from AV (wildcard exception added in Sophos Central). The problem seems to be due to something else that SPL is doing.
We've demonstrated that Sophos SPL is the cause of this problem by running the following test across multiple servers:
a. While SPL is running, any attempt to unmount the Veeam temp filesystem fails.
b. After stopping SPL, unmounting the Veeam temp filesystem succeeds without any issues.
So this seems to suggest that SPL is placing some kind of lock on the Veeam temp filesystem, and this is preventing Veeam from unmounting the temp filesystem. That is causing each server backup to fail.
Since we do not understand the inside workings of all of the SPL modules/features, we're hoping that you might have some insight into why SPL is locking the above-mentioned temporary filesystems?
We've tried using lsof and fuser to identify the offending process, but strangely nothing is showing up. Our working theory is that Sophos SPL is doing something to "lock" the above filesystem, and is thereby preventing it from being unmounted. But we cannot figure out WHAT exactly Sophos SPL is doing that is resulting in this issue.
Thank you in advance!
Hi Abigail Laufer ,
Thank you for reaching out to the Community Forum. Do these servers have a lockdown policy applied? If yes, please try to do the following and see if it helps: In Sophos Central, go to Server Protection > Policies, select the lockdown policy applied, then Add allowed file/folder. You can disregard these steps if there are no lockdown policies in place.
You also mentioned that the filesystem is already exempted using a wildcard exception. If you haven't already, you may also take a look at the following Veeam article, specifically under "Antivirus Software on Linux" section. This article mentions that common issues occur when Antivirus has been configured to tightly secure the /tmp/ directory, which in turn causes conflicts with Veeam's use of the path /tmp/Veeam/. It also lists some preliminary folders and executables that you may add to the exclusions.
Let me know how it goes. Thank you.
Thank you for your reply! We checked with our Sophos Central admins and there are no lockdown policies being applied, and they also added all of the exclusions mentioned in the Veeam KB article. However, it did not resolve the problem. We've opened a case with Sophos Support and have submitted logs for analysis. We haven't heard from them in a couple of days, so I asked for an escalation if possible. Hopefully Sophos Support will be able to provide feedback on the issue. Until that happens, we've have to disable Sophos on all of our Linux servers due to the software conflict. At the core, this is a problem with Sophos SPL, which can be demonstrated by the fact that the issue goes away when we stop Sophos SPL. So we hope that Sophos Support can tell us which component in Sophos SPL is causing this problem, and how best to resolve it.
Per the last update on your open case, our Support team is reviewing the submitted logs. I will follow up with our internal team and get back to you.