Checking disk space consumers on our windows servers we see that Sophos folder is huge on them:
%programdata%\sophos\endpoint defense\data\event journals\sophosED
Some of the subfolders contain tenthousands of files (e.g. Dns or FileBinaryChanges sub-folder) , some folders contain files, that are more than 2 years old.
Who is doing the housekeeping here? That mess slows down backups and other tasks.
one server:
other server:
There will be an option to configure this amount in Central in the future.
But the max value should be 5GB. See: https://docs.sophos.com/central/customer/help/en-us/ManageYourProducts/GlobalSettings/EventJournals/index.html
You could create a case, so somebody could look into this, why your logs are increasing and not rotate correctly.
__________________________________________________________________________________________________________________
thanks LuCar Toni for that information.
I cannot find a setting "Event Journals" in our Sophos Central Management. Our MDR license matches ("You can only configure event journal sizes if you have an XDR, MDR, or MDR Complete license.")
Is the missing feature not implemented yet? You wrote it will come in the future, but the doc reads as it should already be there.
It will be enabled in the future, The online help is already there.
thank you. I will ask support if there is something smart we can do else than deleting the old files manually.
edit: case 06054109
Event Journal configuration is now there in our Central Settings:
It can only control size, not max number of files or retention time.
The documentation* writes, there is a default of 90 days retention. As we can see from here, several folders store data for multiple years or more likely never delete old files and store files from the beginning of the Intercept-X installation on. Probably someone forgot to program that feature for every sub folder.
* = We store event journals on your managed Windows devices. They record activity on your devices, and you can query them with Live Discover "Endpoint" queries in the Threat Analysis Center. See Live Discover. By default, we store 90 days of activity.
Answer from Support was:
The age of data is not relevant. It's all down to the disk space. It could contain data for years, or weeks. It all depends how much data is being written to the Event Journals. The 90 days is based on typical usage/data being written to the Journals - doesn't mean it's the same for every device though.As per the notification with the new release for the Event Journal, the default size is 5250 MB. It doesn't matter how old the data is as long as it doesn't breach that limit.
For me it means, that now with the new feature in Central, the solution to remove old files on all servers once is to temporarily lower the disk space from 5250 MB to something low. And then raise it again. One would need to test this really deletes the files accross all folders (like the Dns folder), not only the ones that already seem have a working file retention enabled by now. It that works, this would need to be done every now and then.
SEDService.exe ("Sophos Endpoint Defense Service") performs the housekeeping of these files and runs every 4 or 5 mins to perform this task. It compresses the current .bin file to .xz for example and keeps the size of each journal under the max size which are defined here:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Sophos Endpoint Defense\EventJournal\Subjects
You can see the max size of each journal subject and if you total up all the values:
Name Property---- --------DirectoryChanges MaxDiskUsageMB : 219Dns MaxDiskUsageMB : 219FileBinaryChanges MaxDiskUsageMB : 437FileBinaryReads MaxDiskUsageMB : 437FileDataChanges MaxDiskUsageMB : 219FileDataReads MaxDiskUsageMB : 219FileHashes MaxDiskUsageMB : 219FileOtherChanges MaxDiskUsageMB : 219FileOtherReads MaxDiskUsageMB : 219FileProperties MaxDiskUsageMB : 219Http MaxDiskUsageMB : 219Image MaxDiskUsageMB : 219Ip MaxDiskUsageMB : 219Network MaxDiskUsageMB : 219Process MaxDiskUsageMB : 437ProcessProperties MaxDiskUsageMB : 219Registry MaxDiskUsageMB : 219System MaxDiskUsageMB : 219Thread MaxDiskUsageMB : 219Url MaxDiskUsageMB : 219WinSec MaxDiskUsageMB : 21918*219 + 3*437 = 3942+1311= 5253 MB.---
In Central you could half this size, e.g.
To:
Once the policy arrives at the client:C:\ProgramData\Sophos\Management Communications System\Endpoint\Cache\CORC37.policy <eventJournalSizeLimit>2625</eventJournalSizeLimit>The registry values are changed to:Name Property---- --------DirectoryChanges MaxDiskUsageMB : 109Dns MaxDiskUsageMB : 109FileBinaryChanges MaxDiskUsageMB : 219FileBinaryReads MaxDiskUsageMB : 219FileDataChanges MaxDiskUsageMB : 109FileDataReads MaxDiskUsageMB : 109FileHashes MaxDiskUsageMB : 109FileOtherChanges MaxDiskUsageMB : 109FileOtherReads MaxDiskUsageMB : 109FileProperties MaxDiskUsageMB : 109Http MaxDiskUsageMB : 109Image MaxDiskUsageMB : 109Ip MaxDiskUsageMB : 109Network MaxDiskUsageMB : 109Process MaxDiskUsageMB : 219ProcessProperties MaxDiskUsageMB : 109Registry MaxDiskUsageMB : 109System MaxDiskUsageMB : 109Thread MaxDiskUsageMB : 109Url MaxDiskUsageMB : 109WinSec MaxDiskUsageMB : 109Next time SEDService.exe carries out its work, the max size per subject should be met.
cool - thank you for that detailled information. as always very good input from you 930!
So we could work around with lowering eventJournalSizeLimit to something and raise it up after one or two days.
a perfect combination would be an option in Central to set the number of days to keep for the Event Journals besides the max amount of disk space consumed.
Yes, it could be there was some event that caused a lot of data to be journaled, reducing the size to purge the journals and then increasing it again should work. It will be destructive but could be done if needed to save space.