Docker builds are five times slower when the antivirus is running

Apologies an advance if this isn't the clearest post. I'm just a Linux user, not a sysadmin, so not sure exactly which product we're using or how it was installed, and I don't have access to any of the central settings or dashboard.

 

We use Docker and have noticed that Docker image builds have slowed down a lot since the antivirus was installed. A build which took 5 minutes previously now takes 25 minutes. Disabling the antivirus with systemctl stop sav-protect allows the build to complete in the shorter time again.

The systemd service that runs the antivirus is named sav-protect and the processes that are busy during the docker build are mtdd and savscand. The slowest part of the build is where Docker copies a load of small files from one intermediate stage of the build to another. I don't know where these files are stored since it's all internal to Docker at that point.

The antivirus is set to run with a niceness of 19 and IT tell me that our Docker data-root (/var/lib/docker) where I would expect these files to be stored is excluded from real-time and scheduled scans.

Can we do anything to prevent the antivirus from interfering with the Docker build? Is there a way to identify which files it's scanning and tell whether the exclusion rules are working correctly? I've looked at a few log files available on the workstation but can't see any record of what is being scanned.

 

$ savscan --version
SAVScan virus detection utility
Copyright (c) 1989-2020 Sophos Limited. All rights reserved.

System time 11:31:51, System date 14 July 2020

Product version : 5.74.0
Engine version : 3.79.0
Virus data version : 5.76
User interface version : 2.03.079
Platform : Linux/AMD64
Released : 23 June 2020
Total viruses (with IDEs) : 51916638

  • Hi  

    I am not sure but please check this article and see if it helps? 

  • In reply to Shweta:

    Thanks for the link  

     

    I think we already use fanotify. 

    $ sudo /opt/sophos-av/bin/savconfig get PreferFanotify
    true

    The article says "Fanotify does not offer on-access scanning within the Docker containers, just the host platform" and "it does not have a negative impact on files and programs running in containers" which is great but I am still seeing a negative impact on my Docker builds.

    Possible reasons:

    • Maybe I'm still using Talpa instead of Fanotify. Are there other commands that can confirm that Talpa is fully disabled?
    • Maybe the Docker build is modifying files on the host and these are getting scanned. That wouldn't be surprising since ultimately all files are stored on the host in some form, but I don't know where these files are. Can the antivirus be told to log which files are being scanned?

    The article talks about running containers as opposed to the process of building a Docker image. I think that builds take place inside (temporary) containers but I'm not familiar enough with low-level Docker to be sure or to know if it makes a difference here.

  • In reply to Carl Fischer:

    Maybe we're not using Fanotify after all.

    $ sudo /opt/sophos-av/bin/savconfig disableTalpa
    $ sudo /opt/sophos-av/bin/savconfig disableFanotify
    true
    $ sudo /opt/sophos-av/bin/savconfig preferFanotify
    true

    It's possible that if Fanotify is disabled then the expressed preference has no effect.

    This can be changed with these commands.

    $ sudo /opt/sophos-av/bin/savconfig set disableTalpa true
    $ sudo /opt/sophos-av/bin/savconfig set disableFanotify false
    $ sudo /opt/sophos-av/bin/savconfig set preferFanotify true

    And confirmed with these.

    $ sudo /opt/sophos-av/bin/savconfig disableTalpa
    true
    $ sudo /opt/sophos-av/bin/savconfig disableFanotify
    false
    $ sudo /opt/sophos-av/bin/savconfig preferFanotify
    true

    And now the Docker builds are running in the expected time. Fantastic. Thanks for the pointer to that article. 

  • In reply to Carl Fischer:

    Hi  

    Glad to know that it is working for you now.