File scanning causing very large data files and RAM usage (unpaged pool)

Hi Jeremy Roberts ,

Thank you for reaching out to the Sophos Community forum.

I can see you posted a similar concern in this thread. This is Sophos Central managed, right? I will be moving your post to the correct forum.

Can you please confirm which Core Agent version are you running? Can you also share a screenshot of the specific service/s that are consuming high RAM usage?

Yes to Sophos Central managed
Currently on Core Agent version 2023.2.2.1

The initial question was what was causing the DB file growing so massive, this was fixed mostly by an update to the latest version of Core Agent , it is not perfect by a country mile but appears better (4-5GB) than it was (12-14GB) for unpaged Pool usage by Sophos

So we still have an issue and probably need to exclude something from being scanned on these particular servers but i am uncertain as to what and because there is no information I can find as to what info is stored in the DBs and how it is generated or what I could look at to get a hint as to what is causing the DBs files and thus the Unpaged pool usage to be so high on these servers. 
I assuming the reason the DB files are stored in unpaged memory is for speed, which seems reasonable.
However I also assume the intention is not to have such large DB files, on most systems these are only a few 100 MB not 4 or 5 GB

One issue is that Non-Paged Pool Memory is not properly reported in Windows Task Manager so you need to use a command line tool such as poolman to see real usage.
Even on a lightly used server I have the Sophos process SG01 using 650MB of unpaged pool RAM.

Back to the point though, This question is specifcaly about finding out what is getting Sophos excited on these Web Servers for a specific Application and trying to work out if I can/should exclude something from scanning  or if they have something on them doing something stupid (knowing the developer not entirely impossible).
Does Sophos log something somewhere or is there some way to find out what info is in the DB files so I a can exclude or stop what ever activity is causing the files to grow so big.
Below is what can be seen in Poolman on one of the afflicted servers

You can see SG01 and SG03 have 3.7GB of unpaged Pool Ram used which is not an optimal situation really.

When you refer to the db files, I assume you are referring to those under: "C:\ProgramData\Sophos\Endpoint Defense\Data\Data Content Records\"?  Can you paste the output of the following so we can see the size of each?

dir 'C:\ProgramData\Sophos\Endpoint Defense\Data\Data Content Records\'

To see what is being scanned, the easiest way is to probably enable Debug logging for "Scan Summaries"

This will create CSV files under: C:\ProgramData\Sophos\Sophos File Scanner\Logs\ which can be used to work out what is being scanned.  That said, if SophosFileScanner.exe isn't busy, it may not be scanning that is the issue.  Just that data content records, which are loaded into non-paged pool is the issue.  But which are the problem files?  CachingKeyDbFull, FileChecksumDbFull, FilePropertyDbFull.

It maybe that these IIS servers are constantly creating some sort of PE files that are being added to the FileChecksumDbFull for example, if so excluding those paths should prevent it growing.

Thanks.

When you refer to the db files, I assume you are referring to those under: "C:\ProgramData\Sophos\Endpoint Defense\Data\Data Content Records\"?  Can you paste the output of the following so we can see the size of each?

dir 'C:\ProgramData\Sophos\Endpoint Defense\Data\Data Content Records\'

To see what is being scanned, the easiest way is to probably enable Debug logging for "Scan Summaries"

This will create CSV files under: C:\ProgramData\Sophos\Sophos File Scanner\Logs\ which can be used to work out what is being scanned.  That said, if SophosFileScanner.exe isn't busy, it may not be scanning that is the issue.  Just that data content records, which are loaded into non-paged pool is the issue.  But which are the problem files?  CachingKeyDbFull, FileChecksumDbFull, FilePropertyDbFull.

It maybe that these IIS servers are constantly creating some sort of PE files that are being added to the FileChecksumDbFull for example, if so excluding those paths should prevent it growing.

Thanks.

Thank you for detailed response, pictures always help.

The files that are causing the issue are the 3 FilePropertyDbFull ones which are around 1GB each and FileContentDbFull which are around 200MB.
Previously the PropertyDb files were up to 3-4GB each and were using 12GB of RAM before the update to the latest core version.

I have previously put an exclusion in for the SplunkD process, in the erroneous belief that this would stop Sophos getting excited about that process and any files it opens, but it would seem that the Splunk log files are comprising 99% of file activity in the csv file, we have excluded .logs as an extension but it would seem that this is not having the desired effect and these log files are still being scanned every time Splunk touches them.
I have added the path to the Splunk log file location and this seems to have calmed things down a bit, certainly less being logged.

Should I expect to see the size of the DB file reduce or do I need to clear down the DB files manually and let them regenerate (hopefully much smaller)?

If you think the exclusions are now in place, I would clear the contents of the DCR. I would do as follows with Tamper off

Stop the following services:

net stop "Sophos File Scanner Service"
net stop "Sophos Network Threat Protection"
net stop "Sophos System Protection Service"
net stop sntp
Then unload the SophosED.sys driver by running
fltmc.exe unload "sophos endoint defense"

Delete the contents of the DCR directory. Ideally reboot or you could:

fltmc.exe load "sophos endoint defense"
net start "Sophos File Scanner Service"
net start "Sophos Network Threat Protection"
net start "Sophos System Protection Service"

Note: The sntp driver will be started when the "Sophos Network Threat Protection" service is started as is depends on it.

Does they still grow rapidly?  Non-paged pool will certainly be lower.

this is all now fixed for me, non-paged pool is now running at 0.7-1GB which is more reasonable.
Thanks for your help.

Thanks for the update.  Worth keeping an eye on it over the next few weeks, it will grow to some extent, inline with the size of the files. Thanks.