File scanning causing very large data files and RAM usage (unpaged pool)

Question

We have group of web servers with very large Unpaged Memory used by Sophos.

Despite the promise of a fix in the latest agent, all that has happened is the RAM usage gets cleared down on a reboot, it then creeps back up at a rate of 2GB a week. By the time monthly patching occurs we have between 8 and 10 GB of memory reserved for Sophos which is a bit rubbish.

Does anyone know of way we can find out what is causing Sophos to get so excited? Specifically which files or processes is causing the FilePropertyDbFull-xxxxxxxxxxxxxxxxxxxxxxxx.bin files to grow so large, I am assuming it is something to do with the file scanner and seeing a large number of files but which ones and what location? as far as I can ascertain the servers are just web servers and there is little or no files transfering through these servers.

Sophos as product does seem to hoard info and is not very willing to share it with anyone.

Added Tags
[edited by: GlennSen at 8:24 AM (GMT -7) on 3 Jul 2024]

Sophos User930 · Accepted Answer

When you refer to the db files, I assume you are referring to those under: "C:\ProgramData\Sophos\Endpoint Defense\Data\Data Content Records\"? Can you paste the output of the following so we can see the size of each? 
 dir 'C:\ProgramData\Sophos\Endpoint Defense\Data\Data Content Records\' To see what is being scanned, the easiest way is to probably enable Debug logging for "Scan Summaries" 
 This will create CSV files under: C:\ProgramData\Sophos\Sophos File Scanner\Logs\ which can be used to work out what is being scanned. That said, if SophosFileScanner.exe isn't busy, it may not be scanning that is the issue. Just that data content records, which are loaded into non-paged pool is the issue. But which are the problem files? CachingKeyDbFull, FileChecksumDbFull, FilePropertyDbFull. 
 It maybe that these IIS servers are constantly creating some sort of PE files that are being added to the FileChecksumDbFull for example, if so excluding those paths should prevent it growing. Thanks.

Sophos User930 · Answer

If you think the exclusions are now in place, I would clear the contents of the DCR. I would do as follows with Tamper off Stop the following services: 
 net stop "Sophos File Scanner Service" net stop "Sophos Network Threat Protection" net stop "Sophos System Protection Service" net stop sntp Then unload the SophosED.sys driver by running fltmc.exe unload "sophos endoint defense" Delete the contents of the DCR directory. Ideally reboot or you could: fltmc.exe load "sophos endoint defense" net start "Sophos File Scanner Service" net start "Sophos Network Threat Protection" net start "Sophos System Protection Service" 
 Note: The sntp driver will be started when the "Sophos Network Threat Protection" service is started as is depends on it. 
 Does they still grow rapidly? Non-paged pool will certainly be lower.

File scanning causing very large data files and RAM usage (unpaged pool)

Top Replies