sav-protect service breaks system updates on Ubuntu

Hello UcP-88AePFWvnSi,

I don't question that the updates fail and disabling sav-protect might be a temporary workaround. I'm always wary though of so-called solutions when the terms used in the description are not quite correct (and thus likely the underlying cause has not been identified). It's On-Access, not On-Demand. It used to work for quite some time and it's not sav-protect that has changed. 
As there are two possible interception methods for On-Access scanning, Talpa or Fanotify, the first question is - which one is used? I cede further questions and comments to DouglasLeeder.

Christian 

Hello UcP-88AePFWvnSi,

I just followed the terminology of another user
I see :).

some specific configuration
can't say what the underlying cause is but it's very likely not some mis-(ing )configuration. As mentioned is this thread the first question is Fanotify or Talpa? Furthermore, if using Talpa Operation Not Permitted normally indicates blocked access because the file is infected. It might be a False Positive - but then the SAV log should have a corresponding entry.

Christian

Hello UcP-88AePFWvnSi,

I just followed the terminology of another user
I see :).

some specific configuration
can't say what the underlying cause is but it's very likely not some mis-(ing )configuration. As mentioned is this thread the first question is Fanotify or Talpa? Furthermore, if using Talpa Operation Not Permitted normally indicates blocked access because the file is infected. It might be a False Positive - but then the SAV log should have a corresponding entry.

Christian

Ok, I think I'm getting somewhere. I'm 'using' Talpa, not Fanotify:

sudo /opt/sophos-av/bin/savconfig get PreferFanotify
false

However, I occasionally get the following notifications from Sophos AV:

An event happened on the computer .....

Unable to load Talpa modules.

Sophos logged an event

So, I checked that Talpa is installed/compiled correctly using the instructions. However, I couldn't get it to compile:

[Talpa-select]
Copyright 1989-2019 Sophos Limited. All rights reserved.
Mon Dec 16 13:17:54 2019 GMT
Linux distribution: [ubuntu]
Product: [Ubuntu 19.10]
Kernel: [5.3.0-24-generic]
Multiprocessor support enabled.
Searching for source pack...
Searching for suitable binary pack...
No suitable binary pack available.
Preparing for build...
Extracting sources...
Configuring build of version 1.25.2...
Building...
Error: Failed to build from source.

The talpaselect.log file says:

[Talpa-select]
Copyright 1989-2019 Sophos Limited. All rights reserved.
2019-12-16 13:17:54 GMT /opt/sophos-av/engine/_/talpa_select select
Linux distribution: [ubuntu]
Product: [Ubuntu 19.10]
Kernel: [5.3.0-24-generic]
Multiprocessor support enabled.
Searching for source pack...
Searching for suitable binary pack...
No suitable binary pack available.
Preparing for build...
Extracting sources...
Configuring build of version 1.25.2...
configuring checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking how to create a ustar tar archive... gnutar
checking whether to enable maintainer-specific portions of Makefiles... no
checking for gcc... gcc
checking for C compiler default output file name... a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables...
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ANSI C... none needed
checking for style of include used by make... GNU
checking dependency style of gcc... none
checking whether gcc and cc understand -c and -o together... yes
checking for ld... ld
checking for egrep... grep -E
checking whether ln -s works... yes
checking for cat... /usr/bin/cat
checking for cut... /usr/bin/cut
checking for sed... /usr/bin/sed
checking for uname... /usr/bin/uname
checking for rm... /usr/bin/rm
checking for xargs... /usr/bin/xargs
checking for Talpa version... 1.25.2
checking for operating system... Linux
checking for kernel headers layout... /lib/modules/5.3.0-24-generic/build/include:/lib/modules/5.3.0-24-generic/build/arch/x86/include:/lib/modules/5.3.0-24-generic/build/arch/x86/include/generated:/lib/modules/5.3.0-24-generic/build/include/generated/uapi:/lib/modules/5.3.0-24-generic/build/arch/x86/include/generated/uapi:/lib/modules/5.3.0-24-generic/build/include/uapi:/lib/modules/5.3.0-24-generic/build/arch/x86/include/uapi
checking for linux/version.h... yes
checking for linux/magic.h... yes - /lib/modules/5.3.0-24-generic/build/include/uapi
checking for uapi/linux/magic.h... yes - /lib/modules/5.3.0-24-generic/build/include
checking for linux/uidgid.h... for uidgid strict type checking header
checking for linux/compiler.h... yes - /lib/modules/5.3.0-24-generic/build/include
checking for uapi/asm/unistd.h... yes - /lib/modules/5.3.0-24-generic/build/arch/x86/include
checking for asm/unistd_64_x32.h... yes - /lib/modules/5.3.0-24-generic/build/arch/x86/include/generated
checking for kernel configuration... checking for linux/kconfig.h... yes - /lib/modules/5.3.0-24-generic/build/include
done
checking for retpoline... configured, no check needed
checking for compilation environment... ok
checking for kernel architecture... x86_64
checking for kernel version code... 328458
checking for kernel version string... 5.3.0-24-generic
checking for RHEL release code... not found
checking for linux/sched.h... yes
checking for unused task flag... 0x1
checking for -m32 build... cannot compile
checking for -mx32 build... cannot compile
checking for System.map... /boot/System.map-5.3.0-24-generic
checking for printk address... 0xffffffff81109293
checking for do_truncate address... 0xffffffff812c6390
checking for linux/fs.h... yes
checking for do_truncate type... type 3
checking for linux/fs.h... yes
checking for vfs_unlink type... with inode
checking for linux/string.h... yes
checking for strndup_user... present
checking for tasklist_lock export... not available
checking for tasklist_lock address... 0xffffffff826050c0
checking for linux/uaccess.h... yes
checking for probe_kernel_read... present
checking for appropriate build system... 2.6 build system detected
checking for linux/dcache.h... yes
checking for __d_path prototype... available
checking for exported __d_path... undetectable
checking for linux/dcache.h... yes
checking for 2.6.38 style locking... post 2.6.38 style locking
checking for __d_path address... 0xffffffff81308690
checking for __d_path type... struct path
checking for vfsmount and br lock... vfsmount lock is mount_lock seqlock
checking for __lookup_mnt_last address... not found
checking for __lookup_mnt address... 0xffffffff812f24f0
assuming system does not have get_fs_root_and_pwd
checking for linux/mount.h... yes
checking for vfsmount mnt_namespace element... assuming vfsmount has mnt_ns
checking for nested mutexes... present
checking for f_dentry in fs struct member... not detected
checking for smbfs... not present
checking for system call table hooking support... yes; shadow mapping
checking for LSM support... disabled
checking for exported hrtimers... missing
checking for struct filename... present
checking for asm-generic/fcntl.h... yes
checking for correct getname... passed
checking for securityfs support... present
checking for binary sysctl support... disabled
checking for legacy configuration support... included
checking for IMA... present
checking for ima_path_check... not present
checking for putname... putname present but not exported
checking for getname... getname present but not exported
checking typedef ctl_table... not detected
checking for X workaround... enabled
configure: creating ./config.status
config.status: creating makefile
config.status: creating clients/Makefile
config.status: creating tests/Makefile
config.status: creating tests/modules/makefile
config.status: creating tests/benchmark/Makefile
config.status: creating config.h
config.status: executing depfiles commands

Building...
Traceback (most recent call last):
File "talpa_select.py", line 2216, in _action
File "talpa_select.py", line 845, in select
File "talpa_select.py", line 1736, in select
File "talpa_select.py", line 1820, in build
File "talpa_select.py", line 1973, in __try_build
SelectException: exc-build-failed

It appears that Talpa is not supported/working on Ubuntu 19.10/the 5.3.0-24-generic kernel. No binary packs are available for it and I can't get it to compile. So, I checked whether Fanotify is supported on my system. It is:

> cat /boot/config-5.3.0-24-generic | grep FANOTIFY
CONFIG_FANOTIFY=y
CONFIG_FANOTIFY_ACCESS_PERMISSIONS=y

I've now enabled it following the instructions:

sudo /opt/sophos-av/bin/savconfig set PreferFanotify true
sudo systemctl restart sav-protect.service

I wonder whether a broken Talpa was the problem. I'll give Fanotify a try for a while and see if matters improve. Thanks.

Hello ,

AFAIK kernel 5.3 is not yet supported, no surprise it doesn't compile.

a broken Talpa
should not get loaded, consequently On-Access shouldn't intercept (or even block) file access.  Wonder why you got the Operation Not Permitted. I don't think DisableFanotify is false after install, i.e. it doesn't automatically fall back to Fanotify when Talpa is not available - or maybe it does but retries to compile/load Talpa unless DisableFanotify is true.
Did you check whether Fanotify was already enabled in savconfig?

Christian

DisableFanotify is false on both machines.

When I first installed Sophos, I just did the various checks suggested in Section 3 of the Configuration Guide to satisfy myself that it was working and it was. So, something was working. I've just checked my savlog. Ahead of every update I have:

talpa.startup        Unable to load Talpa modules.
savd.daemon       On-access scanning enabled using fanotify.

So, it looks like Fanotify was previously being used as a fallback because Talpa couldn't be compiled. If so, explicitly enabling Fanotify by default - which I have just done, is unlikely to have helped, and it doesn't explain why I have been having issues with updates.

I'm afraid we have been some people have problems with fanotify and installing updates (particularly new kernels). Unfortunately no one has raised it at high enough priority with Support, so we haven't been able to allocate any time to investigate the problem.

Thank you Douglas. It would be great if I could get something working and working reliably on Ubuntu 19.10. I don't mind whether it's Fanotify or Talpa. If there's a problem with Fanotify itself, them I'm guessing that would be more difficult for you to address, because it's not your product. So, maybe getting Talpa to compile is the way to go. I've just found the source code repository for Talpa, and I see that someone has opened an issue regarding Linux kernel 5.2 support. That issue also addresses Ubuntu 19.10/kernel 5.3 support and contains a patch that supposedly works albeit throwing some compiler warnings. Possibly, the next step for me ought to be to try that. I note however, that no-one from Sophos has been assigned to that issue. If Sophos is able to allocate some time to this, perhaps updating Talpa to support newer Linux kernels would be the most effective route to ensuring that everyone - even those using newer kernels - has access to reliable on-demand scanning. Thanks.

DouglasLeeder said:

... Unfortunately no one has raised it at high enough priority with Support, so we haven't been able to allocate any time to investigate the problem.

 

Just a quick update that I tried and failed to compile Talpa. It seems that the installation instructions/documentation provided with the source code is out of date and that certain build tools are required that I don't have or which are not set up correctly. I eventually had to give up on this.

However, since the last post my system has been broken two more times during kernel updates due to this issue, requiring me to disable the sav-protect service and then repair the broken packages and dependencies. This state of affairs couldn't continue, so I finally implemented a workaround that disables sav-protect on-demand scanning during apt software updates only, until the issue is resolved. This workaround means putting full trust in any software updates (automatic or otherwise) via the apt package manager. The workaround seems to be effective at preventing broken updates.

I did this by creating the file /etc/apt/apt.conf.d/00sav-protect with the contents:

DPkg::Pre-Invoke {
  "echo Stopping sav-protect...";
  "systemctl stop sav-protect.service";
  "echo Done";
};
DPkg::Post-Invoke {
  "echo Starting sav-protect...";
  "systemctl start sav-protect.service";
  "echo Done";
};

I hope this might be helpful for others encountering the same issue.

Yes, paid support, via phone calls to the Support number.

If enough people do that, it would get raised up our priority order.

Sounds good. Thanks.

UcP-88AePFWvnSi said:

I did this by creating the file /etc/apt/apt.conf.d/00sav-protect with the contents:

This is a workaround that can help me as well, thanks a lot!