Running sav-protect/update in Ubuntu 16.04 docker container. Problem: "WARNING: No OS Version"

Hello Jean-Sebastien Legare,

I see you've figured out some of the install.sh arguments. In addition you'd need at least the -update-source-username and -update-source-password (and note that when properly installed it would update automatically, no need to run savupdate with a cron job). I'd suggest that you use mkinstpkg --sophos (assuming you're updating from Sophos) to create an install package.

Christian

Thank you,

I've successfully generated a .deb file using mkinstpkg. Thanks for letting me know about "mkinstpkg --sophos". I assume my credentials are hardcoded inside the .deb at this point ?

The --update-source-username=USERNAME --update-source-password=PASSWORD require passing a username/password combo. Is there a way to generate one on the fly?

Also, what is it exactly that triggers the savupdate in cases where live protection is disabled? I see the update period present in the savd.cfg, but I'm not sure which of the daemons takes care of triggering the update process. In my docker container sav-protect is not running, and the init scripts are not exactly working, due to missing dependencies.

Also, does install.sh --update-all-distros=False affect the number of virus definitions received in an update (my linux server scans files consumed by various client OS, so I need all definitions)? -- or is it only about the program files ?

Hello Jean-Sebastien Legare,

my credentials are hardcoded ... generate one on the fly?
yes. Naturally for the licensed version you have fixed credentials, if you select free when prompted mkinstpkg contacts Sophos (I assume to obtain free credentials), haven't run though this procedure though so I can't say hwat's in the package.

what is it exactly that triggers the savupdate in cases where live protection is disabled
some clarification: Live protection means that the scanner and other components make lookups to fetch the latest information on potential threats. It's not for updating the static protection data. Scheduled (periodic) updates - detection data as well as software - are, unless automatic updating is disabled, triggered by savd according to the settings in savd.cfg.

does install.sh --update-all-distros=False affect the number of virus definitions received ... I need all definitions
no. First of all scanning engine and definitions are the same for all products and platforms. --update-all-distros refers to the Talpa Binary Packs - by default an endpoint downloads only the one suitable for its distribution and kernel - and only comes into play when the endpoint acts as an update server for other Linux endpoints.

Christian

I may have confused live protection and on-access scanning.

For my use-case, I just want to scan files on-demand -- scan file attachments from incoming email. I'd also like to detect vulnerabilities that are recent. So what's the minimum I need?

I don't have savd running in my container (it just won't start in a docker container), `savdctl enable` and `/opt/sophos-av/engine/savd` both fail miserably.

However, I've installed savdid (the dynamic interface daemon), and that's running fine (although it consumes 239M of resident memory). It's hooked up on a unix socket, and I've configured amavisd-new to pass mail through it. I've scanned a few different emails with it, including some with eicar, and it can detect viruses.

My plan was to run savupdate in a cron job (since I'm not able to run any other sav* daemons) to get up-to-date virus definitions, and send savdid a SIGHUP after an update so it loads the new IDEs before handling the next request.

Would that do it?

===

Also, For reference: the output of mkinstpkg. I confirm that the credentials are hardcoded in the deb at the time mkinstpkg is called. Running the command a second time resulted in the same credentials as in the first attempt however, so there is some caching at play I'm guessing.

root@host:/opt/sophos-av/update# ./mkinstpkg -o 1.deb --host -D --sophos
... Press to display Licence. Then press  to scroll forward. ... 
[including the GNU Public Licence.] 
Do you accept the licence? Yes(Y)/No(N)  > Y 
Where do you want to install Sophos Anti-Virus? [/opt/sophos-av] > 
Do you want to enable on-access scanning? Yes(Y)/No(N) > N
On-access scanning disabled. Use savscan for on-demand scanning. Updating directly from Sophos. 
Do you wish to install the Free (f) or Supported (s) version of SAV for Linux? [s] > f
The Free version of Sophos Anti-Virus for Linux comes with no support. 
Forums are available for our free tools at http://openforum.sophos.com/
Do you need a proxy to access Sophos updates? Yes(Y)/No(N) > N
Fetching free update credentials. 
Commandline: --enableSavProtectOnBoot=True --update-free=True --update-source-username=XXX --update-source-password=YYY --instdir=/opt/sophos-av --update-cache-path=/opt/sophos-av/update/cache/Primary --update-type=s --update-source-path=sophos: --acceptlicence=True --enableRMS=false --automatic=True --enableOnAccess=false --update-period-minutes=60 Deb package is '/opt/sophos-av/update/1.deb'

The produced archive is quite small (5.4M), in comparison with the full installer package (500MB if I recall)

root@host:/opt/sophos-av/update# ls -lh 1.deb 
-r-------- 1 root root 5.4M Aug 29 15:02 1.deb

The deb archive contains a single savinstpkg.tgz which is unpacked in the postinst debian step

root@host:/opt/sophos-av/update# dpkg -c 1.deb 
-r-------- root/root   5600746 2017-08-29 15:02 tmp/savinstpkg.tgz

root@host:/opt/sophos-av/update# ar -p 1.deb control.tar.gz | tar xzf - -O postinst
#!/bin/sh
set -e
cd /tmp
tar -zxf savinstpkg.tgz
cd sophos-av
./install.sh
cd ..
rm -rf sophos-av savinstpkg.tgz

There is a file called installOptions in the package, with the following parameters to drive the install:, including the username and password (edited out):

     --enableSavProtectOnBoot=True \
     --update-free=True \
     --update-source-username=FAVXXXXXXXXX \
     --update-source-password=YYYYYYYYY \
     --instdir=/opt/sophos-av \
     --update-cache-path=/opt/sophos-av/update/cache/Primary \
     --update-type=s \
     --update-source-path=sophos: \
     --acceptlicence=True \
     --enableRMS=false \
     --automatic=True \
     --enableOnAccess=false \
     --update-period-minutes=60

Hello Jean-Sebastien Legare,

I have no experience with Docker containers so forgive me if I say something stupid.
Do I understand correctly that you are running the savdid daemon in a container and that you install SAV when the container is started (savdi as well or is it already in the container)? Guess savscand is also not running. What's the advantage of using a container for savdid?
Updating the virus data with a cron job followed by SIGHUP should do. Dunno what exactly savdid needs from the sav installation (obviously some or all contents of sophos-av/lib, /lib/sav, and /engine).

Christian

My container runs a suite of mailserver software (postfix, amavisd, spamassassin, etc). The main reason for running the antivirus within the same docker container as the rest is simplicity -- all services come up at once -- and everything is local. I could install just the AV in a separate container, but I think I'd have the same problems

You are correct. I run savdid in the same container where I installed sav (using install.sh). The entrypoint to my container starts multiple daemons that keep running, including savdid (and postfix, and a few others). So the container stays up all the time, whether or not the AV is running. Correct, savd and savscand are not running -- they just won't start, probably due to something missing (that the scripts expect) in the docker environment.

My postfix configuration uses Amavisd-new as an MTA plugin content filter. Whenerver postfix receives an incoming email, it passes it to amavisd for analysis. 

Amavisd has a few plugins for antivirus checking (it does spam checking too). Namely, it supports submitting file attachments over a SOPHIE-compliant socket. That's where savdid is useful.

Another option for scanning file attachments with amavis is to use the "sweep" binary that comes with sophosav. This is the command line utility to scan files on demand. This is what I used at first, before I installed savdi, because it was simple. However, on the limited CPU resources I have (the container runs over a t2.micro AWS instance), every invocation of "sweep" takes over 5 seconds -- presumably because it has to pay a high cost for loading up all that python code at each invocation. That 5 second processing time for each incoming mail, was a dealbreaker, so I used the savdi daemon. Once sadvid is up and running the engine stays loaded and warm, and it's always ready to go. It's much faster, I can scan small attachments in milliseconds.

I'd like to help you sort out what it is with the docker environment that prevents savd from running -- I'm happy to make a different thread. The strace logs don't reveal anything obvious. I figure it's a simple assertion or sanity check that's not passing in the code. But short of decompiling the python bytecode, I'm in the dark. The "No OS version" warnings don't exactly inspire confidence that all the right codepaths are taken.

Just for reference, what I'm running is a modified docker image from: https://github.com/tomav/docker-mailserver  . I've replaced ClamAV (open source, free of cost) with SophosAV for Linux (free version). I've found that SophosAV (savdid) has a resident memory footprint (234MB) that is 250MB less than clamav's daemon (clamd, with a whopping 519MB!). This is important, because for my use case everything has to fit within 1Gib of ram. So, 234MB is still a lot, but it's not 519MB. Kudos. I couldn't find a comparison of the performance (cpu, false negative rates) of the two antivirus solutions -- but sophos definitely wins on the memory front.

My container runs a suite of mailserver software (postfix, amavisd, spamassassin, etc). The main reason for running the antivirus within the same docker container as the rest is simplicity -- all services come up at once -- and everything is local. I could install just the AV in a separate container, but I think I'd have the same problems

You are correct. I run savdid in the same container where I installed sav (using install.sh). The entrypoint to my container starts multiple daemons that keep running, including savdid (and postfix, and a few others). So the container stays up all the time, whether or not the AV is running. Correct, savd and savscand are not running -- they just won't start, probably due to something missing (that the scripts expect) in the docker environment.

My postfix configuration uses Amavisd-new as an MTA plugin content filter. Whenerver postfix receives an incoming email, it passes it to amavisd for analysis. 

Amavisd has a few plugins for antivirus checking (it does spam checking too). Namely, it supports submitting file attachments over a SOPHIE-compliant socket. That's where savdid is useful.

Another option for scanning file attachments with amavis is to use the "sweep" binary that comes with sophosav. This is the command line utility to scan files on demand. This is what I used at first, before I installed savdi, because it was simple. However, on the limited CPU resources I have (the container runs over a t2.micro AWS instance), every invocation of "sweep" takes over 5 seconds -- presumably because it has to pay a high cost for loading up all that python code at each invocation. That 5 second processing time for each incoming mail, was a dealbreaker, so I used the savdi daemon. Once sadvid is up and running the engine stays loaded and warm, and it's always ready to go. It's much faster, I can scan small attachments in milliseconds.

I'd like to help you sort out what it is with the docker environment that prevents savd from running -- I'm happy to make a different thread. The strace logs don't reveal anything obvious. I figure it's a simple assertion or sanity check that's not passing in the code. But short of decompiling the python bytecode, I'm in the dark. The "No OS version" warnings don't exactly inspire confidence that all the right codepaths are taken.

Just for reference, what I'm running is a modified docker image from: https://github.com/tomav/docker-mailserver  . I've replaced ClamAV (open source, free of cost) with SophosAV for Linux (free version). I've found that SophosAV (savdid) has a resident memory footprint (234MB) that is 250MB less than clamav's daemon (clamd, with a whopping 519MB!). This is important, because for my use case everything has to fit within 1Gib of ram. So, 234MB is still a lot, but it's not 519MB. Kudos. I couldn't find a comparison of the performance (cpu, false negative rates) of the two antivirus solutions -- but sophos definitely wins on the memory front.

Hello Jean-Sebastien Legare,

kinda heavy-weight container[:)].

every invocation of "sweep" takes over 5 seconds
even with "unlimited" resources initialization takes a significant amount of time, not only the code but also the AV data are loaded and prepared. sweep has to go through the same initialization steps as savscand or savdi so it's for occasional use only. 

Regarding the installation of SAV in a container DouglasLeeder might be able to give you some information - he's the expert. AV in a separate container would IMO better fit the container concept, you'd need only the parts that are in a UNIX installation though.

Christian

I'm afraid we don't support running SAV within a container, so I don't know why you are getting all those errors.

The No OS Version is coming from targetsystem - I guess you don't have lsb_release in your container.

savupdate is normally scheduled by savd, which is started by the sav-protect init script/unit.

There's always some finnicking around to get stuff to work in containers. I'm fine with it. It's just that the errors I'm getting are difficult to diagnose in the absence of system calls (nothing shows up in strace), and entries in logs. I figure it fails early. If you're familiar with the code, the exit code is 51 for many of the daemons, maybe that gives another hint.

In the very first message on this thread, I've listed a `cat /etc/lsb_release` and the output of uname, as run from the container. I do have lsb_release. The container image is "FROM ubuntu:16.04"

the init scripts normally installed don't work very well int the container environment due to the lack of systemd/upstart in the container, as welll as the shell utilities for printing "* STARTING X" with colors and fancy bells.

invoking the daemons directly doesn't work either, so I'd have to make that work first before i cook up a corresponding init script.

I guess please could you show the output of 

# lsb_release -a

Certainly when I launch a ubuntu:16.04 docker image I don't have lsb_release installed.

oops! I missed the '_'

You're correct. the file is there in /etc/lsb-release, but I had to install the package `lsb-release` to get that binary.

root@mx1:/# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.3 LTS
Release: 16.04
Codename: xenial

installing the package got rid of the "No OS Version" warning at least.