Live Discover: Query Cancelled: E Process SophosOsqueryExtension.exe exceeded 30% CPU limit

the VM machine has 2 vCPU - sure - when that process runs, it consumes 50% - what the heck is that 30% limitation? Are you serious, live discover will only run longer than 9 seconds on machines with 4 core CPU??

What happens if you change the query to just:

SELECT    
    STRFTIME('%Y-%m-%dT%H:%M:%SZ', DATETIME(file_journal.time,'unixepoch')) AS date_time,
    CASE file_journal.eventType
        WHEN 0 THEN 'Created'
        WHEN 1 THEN 'Renamed'
        WHEN 2 THEN 'Deleted'
        WHEN 3 THEN 'Modified'
        WHEN 4 THEN 'HardLink Created'
        WHEN 5 THEN 'Timestamps Modified'
        WHEN 6 THEN 'Permissions Modified'
        WHEN 7 THEN 'Ownership Modified'
        WHEN 8 THEN 'Accessed'
        WHEN 9 THEN 'Binary File Mapped'
    END AS event_type,
    REPLACE(file_journal.pathname, RTRIM(file_journal.pathname, REPLACE(file_journal.pathname, '\', '')), '') AS file_name,
    file_journal.pathname AS file_path,
    file_journal.sophosPID AS sophos_pid
FROM sophos_file_journal AS file_journal
WHERE
    file_journal.pathname LIKE '$$file_path$$'
    AND file_journal.time > $$start_time$$
    AND file_journal.time < $$end_time$$
ORDER BY file_journal.time DESC

This just uses the same time frame as you define in the variables but only reads from the sophos_file_journal table.

It might be worth running Process Explorer on the client, with the Performance Graph tab of the SophosOsqueryExtension.exe process open.

If that is still slow, then we can look into the data behind this table.

thanks for your answer.

that's the same with your modified query.

It's just, that the query is stopped after 10 seconds due to the 30% for 9 seconds soft limit.

then it stops.

This would only run if I put 2 more CPU core into that machine. And I think that is not a solution.

To prevent users from causing severe performance degradation, the queries have a CPU and RAM limiter that prevents any one query from locking/degrading a system down to unresponsive. 

Are you looking for things on the machines within the last 24 hours?

Thanks for that, that simplifies the problem somewhat. I wonder how your data is distributed across the archived journal files?

For example if I run a Process Monitor trace while running the modified query, with a filter for paths that end if .xz and .bin, where the process is SophosOsqueryExtension.exe, then create a report with the "Count occurrences" for the path, I get the following:

"Value","Count"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherChanges\FileOtherChanges-0000000003164745-0000000003250d7f-132933840932865956-132933867673898059.xz","218"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryReads\FileBinaryReads-000000000340b8f8-132935705696774197.bin","91"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryChanges\FileBinaryChanges-0000000003183c6d-00000000032972a8-132933848745300477-132933886082289206.xz","90"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherReads\FileOtherReads-000000000316472e-0000000003282e33-132933840890635270-132933877533763127.xz","86"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherReads\FileOtherReads-00000000030cba92-0000000003122f0a-132933691431778441-132933727435204637.xz","60"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherReads\FileOtherReads-0000000003282e7c-00000000032dab03-132933877540154213-132933916382080575.xz","60"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherReads\FileOtherReads-000000000340b8a7-132935705682146621.bin","60"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherChanges\FileOtherChanges-00000000030cba8e-0000000003122ed6-132933691409439966-132933727124201209.xz","54"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryReads\FileBinaryReads-00000000030cba91-0000000003122f19-132933691416625746-132933727485232707.xz","52"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherReads\FileOtherReads-000000000334339d-000000000337ac43-132934042507952658-132934091597705364.xz","50"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherChanges\FileOtherChanges-0000000003250d80-00000000032bbb06-132933867673908032-132933904453521310.xz","42"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryReads\FileBinaryReads-000000000316c3a6-000000000328d5f1-132933843889941531-132933880512936546.xz","41"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherChanges\FileOtherChanges-0000000003123170-0000000003149e94-132933728208560180-132933804781110586.xz","40"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherChanges\FileOtherChanges-000000000340b8c1-132935705689653792.bin","39"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryReads\FileBinaryReads-00000000032dd5a7-00000000033100c4-132933918241352546-132933966088059542.xz","38"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryReads\FileBinaryReads-000000000328d75d-00000000032dd51e-132933880588809428-132933918216194629.xz","34"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherChanges\FileOtherChanges-00000000030ac3a7-00000000030cb982-132933279101539688-132933280184891740.xz","34"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryReads\FileBinaryReads-00000000030414a7-0000000003085ccb-132933222917324032-132933260408351291.xz","32"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherReads\FileOtherReads-00000000032dabb9-00000000033101d1-132933916445576886-132933966687644855.xz","32"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryReads\FileBinaryReads-0000000002ffbf28-00000000030396ab-132933091430769813-132933128067779456.xz","30"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherChanges\FileOtherChanges-0000000003082eda-00000000030ac291-132933257437188130-132933279004435186.xz","30"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherChanges\FileOtherChanges-00000000032bbbea-0000000003310185-132933904541641632-132933966687127990.xz","30"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherReads\FileOtherReads-000000000331132d-000000000334337d-132934006488505601-132934042497631497.xz","30"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryReads\FileBinaryReads-000000000338adc3-00000000033b86f3-132934856572415035-132934913417738966.xz","28"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherChanges\FileOtherChanges-00000000033b87f3-00000000033f1b4d-132934913457479442-132934950592210685.xz","28"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherReads\FileOtherReads-00000000030414a2-0000000003085cc6-132933222917143475-132933260408020495.xz","28"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherReads\FileOtherReads-0000000003149f76-000000000316470f-132933804874354665-132933840869528127.xz","28"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherReads\FileOtherReads-000000000338ae42-00000000033b86a2-132934856574404526-132934913414827354.xz","28"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherReads\FileOtherReads-00000000033b87b1-00000000033f1dc4-132934913447237160-132934950667842504.xz","28"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryReads\FileBinaryReads-00000000030ac3ab-00000000030ca8e4-132933279102476972-132933280144804412.xz","26"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryReads\FileBinaryReads-0000000003311336-00000000033432fe-132934006488661887-132934042433530454.xz","26"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryReads\FileBinaryReads-0000000003343700-000000000337ac36-132934042794612783-132934091457781825.xz","26"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryReads\FileBinaryReads-00000000033b87ca-00000000033f1bd6-132934913456007507-132934950597458806.xz","26"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherChanges\FileOtherChanges-000000000304143e-0000000003082eb2-132933220975626737-132933257409135276.xz","26"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherChanges\FileOtherChanges-0000000003311372-000000000334337a-132934006489087527-132934042496998190.xz","26"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherChanges\FileOtherChanges-000000000334348d-000000000337abee-132934042558282785-132934086277963377.xz","26"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherChanges\FileOtherChanges-000000000338adc5-00000000033b8719-132934856572430185-132934913418197011.xz","24"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherReads\FileOtherReads-0000000003085d26-00000000030ac290-132933260466097417-132933279004435186.xz","24"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryReads\FileBinaryReads-0000000003085ee7-00000000030ac10a-132933260502870349-132933278982370226.xz","22"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryReads\FileBinaryReads-000000000312323a-0000000003149cdf-132933728590057854-132933804580481981.xz","22"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryReads\FileBinaryReads-0000000003149fa4-000000000316c314-132933804899220632-132933843869784729.xz","22"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryReads\FileBinaryReads-000000000337b8b4-0000000003384998-132934187568878154-132934501211811533.xz","22"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherReads\FileOtherReads-0000000003024eb2-0000000003041421-132933113342673521-132933214517390696.xz","22"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryChanges\FileBinaryChanges-000000000314f51b-0000000003183c2e-132933812080114413-132933848743728376.xz","20"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryReads\FileBinaryReads-0000000003384b5b-000000000338adbf-132934501222322796-132934856572324775.xz","20"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherChanges\FileOtherChanges-0000000003024e81-000000000304142a-132933113336299806-132933214519557630.xz","20"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherChanges\FileOtherChanges-0000000003149fae-0000000003164705-132933804899372418-132933840850612466.xz","20"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherChanges\FileOtherChanges-000000000337b8d8-0000000003384b06-132934187569566699-132934501220464832.xz","20"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherReads\FileOtherReads-00000000030ac3ac-00000000030cb8dd-132933279125284071-132933280183485838.xz","20"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherReads\FileOtherReads-0000000003122f78-0000000003149f60-132933727826909560-132933804862844304.xz","20"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryChanges\FileBinaryChanges-00000000030dc3a7-000000000311e205-132933691956932711-132933720977923655.xz","18"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryChanges\FileBinaryChanges-000000000337f4fa-000000000338595c-132934500467515757-132934501284761258.xz","18"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryReads\FileBinaryReads-0000000003039c49-00000000030413fb-132933128367287489-132933206558125766.xz","18"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherChanges\FileOtherChanges-0000000003384b07-000000000338ad6f-132934501220495857-132934856570971646.xz","18"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherReads\FileOtherReads-000000000337b8b3-000000000338490c-132934187568801550-132934501196710269.xz","18"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryChanges\FileBinaryChanges-000000000309d507-00000000030a9616-132933275585410318-132933278864589023.xz","16"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryChanges\FileBinaryChanges-00000000030ac78c-00000000030c6789-132933279170117250-132933279883596039.xz","16"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryChanges\FileBinaryChanges-0000000003125dbc-000000000314652a-132933743087072489-132933802606416139.xz","16"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryChanges\FileBinaryChanges-00000000032b3b39-0000000003306572-132933899867667119-132933936170583138.xz","16"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryChanges\FileBinaryChanges-00000000033068b0-000000000330e5bd-132933936241091969-132933938487176346.xz","16"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryReads\FileBinaryReads-00000000033101c6-0000000003311256-132933966687543759-132934003256892784.xz","16"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryReads\FileBinaryReads-000000000337ac52-000000000337b865-132934094676628434-132934179168359367.xz","16"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryReads\FileBinaryReads-00000000033fe86d-000000000340b879-132934991039647205-132935705655446367.xz","16"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileDataChanges\FileDataChanges-0000000003181646-000000000322448b-132933848493901687-132933863916254424.xz","16"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherChanges\FileOtherChanges-00000000033101fc-000000000331124c-132933966688259075-132934003245511625.xz","16"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherChanges\FileOtherChanges-00000000033fe85b-000000000340b876-132934991038718531-132935705655290187.xz","16"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherReads\FileOtherReads-00000000033101e4-0000000003311255-132933966687862450-132934003256580684.xz","16"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherReads\FileOtherReads-0000000003384c6d-000000000338add1-132934501234082286-132934856572672786.xz","16"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryChanges\FileBinaryChanges-0000000003005f1e-0000000003036bd6-132933095472883285-132933126994235522.xz","15"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryChanges\FileBinaryChanges-0000000003049aee-00000000030652a7-132933224966162634-132933239584973736.xz","15"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryChanges\FileBinaryChanges-0000000003310fbe-0000000003326264-132934002113545697-132934026772588079.xz","15"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryChanges\FileBinaryChanges-000000000335fbc0-0000000003365caf-132934057799845137-132934062984406633.xz","15"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryChanges\FileBinaryChanges-000000000338bcce-00000000033aafc7-132934856628773014-132934878050458260.xz","15"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryChanges\FileBinaryChanges-00000000033b720b-00000000033e3314-132934912235232719-132934939442052630.xz","15"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryReads\FileBinaryReads-00000000033f1e87-00000000033fe859-132934950731618597-132934991038569908.xz","15"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileDataChanges\FileDataChanges-000000000301d964-000000000302f6d1-132933110477677193-132933118037169723.xz","15"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileDataChanges\FileDataChanges-000000000310ec3b-0000000003138012-132933710515625555-132933743859006010.xz","15"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileDataChanges\FileDataChanges-0000000003149641-0000000003149642-132933803957650732-132933803957700830.xz","15"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileDataChanges\FileDataChanges-00000000032ba8ae-00000000032c5256-132933903469702030-132933908703526358.xz","15"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileDataReads\FileDataReads-000000000320a718-000000000320cd1c-132933861581398935-132933861855441797.xz","15"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherChanges\FileOtherChanges-000000000337ac93-000000000337b853-132934094678656416-132934170771486807.xz","15"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherReads\FileOtherReads-000000000337ac50-000000000337b83b-132934094676372583-132934170767686813.xz","15"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherReads\FileOtherReads-00000000033fe881-000000000340b88e-132934991040528990-132935705662103602.xz","15"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherChanges\FileOtherChanges-00000000033f1f28-00000000033fe858-132934950793028440-132934991038544384.xz","14"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherReads\FileOtherReads-00000000033f1e6f-00000000033fe862-132934950710701756-132934991038895114.xz","14"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryChanges\FileBinaryChanges-00000000033f3afd-00000000033f3afe-132934951744302922-132934951744302922.xz","13"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryChanges\FileBinaryChanges-0000000003402411-132935704283670497.bin","13"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileDataChanges\FileDataChanges-000000000339529a-132934862623471307.bin","13"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileDataReads\FileDataReads-00000000032baceb-132933903518449338.bin","13"

For me, the data comes from about 90 different files.  As you go back further in time or choose a bigger time window, you would expect that the extension has to unpack more data from the compressed journal files but I wonder how many files are being accessed to create the results.

It is intensive on disk, reading lots of files but for CPU, is it in the decompression?

If you say, the data is coming from 500 different xz files, that might explain it.

I wonder how these numbers vary for going back 1, 2 and 3 days for example?  Maybe there is a lot of data on a certain day due to an OS update for example?  The file paths contain the "subject", e.g. "FileOtherChanges", "FileOtherReads", etc. to give a clue as to the types of file operations being extracted and also has the timestamps in the filenames.

Beyond understanding that, It might be worth running from an admin prompt:

wpr.exe -start GeneralProfile

Running the query and when it completes, stop the performance capture with:

wpr.exe -stop C:\gp.etl

I'd be interested to see where SophosOsqueryExtension.exe is spending all it's CPU time. Opening that trace in Windows Performance Analyzer would help.  Without the symbols for SophosOsqueryExtension it will be a bit trickier but you can see the APIs being called.  Happy to take a look if needed.

Thanks

Writing to the journals is optimized. Reading is slower. To bridge that gap, we created the Data Lake in our XDR offering. The endpoints upload their journals into the lake and you can process a query against that much faster. The limitation is that the data is not the most up to date - you have to wait for data transmission. 

So, if the data is okay to be a little stale - use the Data Lake. 

Live Queries are for data that you need to be 100% up to date as of execution.

One of the ways to constrain the burden a query like that places on the endpoint - limit it to an hour or two in time range. Then use the Data Lake for data older than that.

: "To prevent users from causing severe performance degradation, the queries have a CPU and RAM limiter that prevents any one query from locking/degrading a system down to unresponsive."

A: I think this is a good setting. But from my point of perspective, there must be a way to disable that limitation directly from the query within Central. If you have a critical situation you would not care about performance - you need results, ASAP!

Put a checkbox to Central Query like "Run query with highest priority. This can cause heavy load on the target machine(s)!"

The target machine is a file server. So surely, there are lots of journal files as mentioned by Sophos User930 . Someone modified a special directory and we don't know when. We want to determine what happened afterwards but cannot.

How can I run that query? I just want to start the query at the evening and I don't care for high CPU load on the file server at night.

Update: Support sent me this KB: https://support.sophos.com/support/s/article/KB-000039257?language=en_US

I think it's outdated because I have 9 seconds, written is 12 seconds. And they mention other soft limits as well. I wonder if this product is designed to work on servers where you will find proably more changes than on endpoints. Especially when your servers have been attacked successfully.

Sophos User930 said:

I wonder how your data is distributed across the archived journal files

In

C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileDataChanges

there are 9123 .xz files beginning with 15.07.2020.

..\FileBinaryChanges

has 17755 files first from 23.12.2019

as written: it's a file server

who's cleaning up that old stuff?

The data is purged but it's based on size not time. Under:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Sophos Endpoint Defense\EventJournal\Subjects

...is a key for each subject, e.g. in this case: FileDataChanges:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Sophos Endpoint Defense\EventJournal\Subjects\FileDataChanges

MaxDiskUsageMB = 150 (dec) 

The SEDService will keep the total for this subject under 150MB.  Some of the others are 300MB, E.g. FileBinaryReads.

This is how you are able to query months worth of data.

You could, in theory, change the 150 to 10, wait 5 mins and a number of xz files will be removed to keep it under the new size specified. This is removing data.  

The files aren't all opened when a query comes in, the file name is enough to hint to the query which files need to be opened so the smaller the timeframe queried for, the less files are opened and decompressed.

good answer to that sub-question. thanks!

unfortunately Sophos Support just came back to me with a link to the system requirements of Intercept-X... which are met by the server  :-(

good answer to that sub-question. thanks!

unfortunately Sophos Support just came back to me with a link to the system requirements of Intercept-X... which are met by the server  :-(

Thanks. In the initial post, you are querying from:29th March to 31st March. I wonder how many .xz files (now it's in the past, all the data will be coming from xz files not .bin) are accessed to cover that 3 day time frame on this server?

Did you manage to run a Process Monitor trace while the query was running with a filter for paths that end if .xz, where the process is SophosOsqueryExtension.exe. Once the query has completed then create a report with the "Count occurrences" for the path. This will give a unique list of file paths. How many is that? Thousands?

If it's a busy server, under the Filter Menu, choose "Drop filtered Events" so only those events that match the filter are retained.

Hi,

it's about 50 files in DataChanges for the 3 days.

C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileDataChanges>dir | find "29.03.2022"
29.03.2022  05:34             1.328 FileDataChanges-00000000d60c996c-00000000d60d7ad7-132929665066719021-132929676050159249.xz
29.03.2022  06:33             1.272 FileDataChanges-00000000d621d75e-00000000d6238299-132929981303360973-132929997867519067.xz
29.03.2022  07:40             3.764 FileDataChanges-00000000d624a2e5-00000000d627cbfe-132930018921075218-132930048676843580.xz
29.03.2022  08:45            11.836 FileDataChanges-00000000d628524f-00000000d62b0d7e-132930058078122819-132930095518865813.xz
29.03.2022  09:50            21.536 FileDataChanges-00000000d62b218b-00000000d62df2e3-132930095888162623-132930134435765231.xz
29.03.2022  10:55            31.640 FileDataChanges-00000000d62df403-00000000d631115a-132930134614349814-132930173475262785.xz
29.03.2022  11:55            34.528 FileDataChanges-00000000d6311286-00000000d633e06e-132930173599434091-132930209522586452.xz
29.03.2022  12:55            34.336 FileDataChanges-00000000d633e119-00000000d636aa96-132930209603746037-132930245518409002.xz
29.03.2022  13:55            22.960 FileDataChanges-00000000d636ac7f-00000000d6394728-132930245733325204-132930281504029499.xz
29.03.2022  14:55            60.604 FileDataChanges-00000000d6394d7c-00000000d63c05c4-132930281765089475-132930317411613050.xz
29.03.2022  16:00            24.768 FileDataChanges-00000000d63c1088-00000000d63ec9ce-132930317861781663-132930355935582385.xz
29.03.2022  17:01            22.808 FileDataChanges-00000000d63ed134-00000000d6447237-132930356621884923-132930393370838740.xz
29.03.2022  18:03            15.320 FileDataChanges-00000000d64486d2-00000000d6482019-132930394573454676-132930429978684584.xz
29.03.2022  19:03             6.868 FileDataChanges-00000000d6482f3d-00000000d64a8c57-132930430938434334-132930464684625087.xz
29.03.2022  21:03             1.092 FileDataChanges-00000000d64ab9c5-00000000d64ce252-132930467939053506-132930499655385313.xz

C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileDataChanges>dir | find "30.03.2022"
30.03.2022  05:44             2.584 FileDataChanges-00000000d64f9ef5-00000000d651d2da-132930541790486129-132930574320526286.xz
30.03.2022  06:44             2.804 FileDataChanges-00000000d6659c23-00000000d667f9f3-132930851165648419-132930887023648277.xz
30.03.2022  07:44             3.128 FileDataChanges-00000000d6681758-00000000d66a17dd-132930887427883352-132930917334586962.xz
30.03.2022  08:49            17.572 FileDataChanges-00000000d66a9494-00000000d66d514c-132930923773825018-132930961091767598.xz
30.03.2022  09:54            33.412 FileDataChanges-00000000d66d5d2a-00000000d670a0c7-132930962199167659-132931000920768761.xz
30.03.2022  10:54            22.564 FileDataChanges-00000000d670a242-00000000d6732ff3-132931001125777481-132931036799623809.xz
30.03.2022  11:59            33.808 FileDataChanges-00000000d673404d-00000000d6760aa6-132931037470637289-132931075942190195.xz
30.03.2022  13:04            26.068 FileDataChanges-00000000d67615d2-00000000d678e2d4-132931076410841190-132931115094116358.xz
30.03.2022  14:09            52.340 FileDataChanges-00000000d678e679-00000000d67baff1-132931115288213230-132931154107772172.xz
30.03.2022  15:12            47.108 FileDataChanges-00000000d67bb020-00000000d67ec2fd-132931154121733531-132931192002749171.xz
30.03.2022  16:14            23.224 FileDataChanges-00000000d67ec613-00000000d682a40c-132931192120150649-132931229177282443.xz
30.03.2022  17:19            17.796 FileDataChanges-00000000d682b788-00000000d685835a-132931230388440284-132931268163032139.xz
30.03.2022  18:19            12.416 FileDataChanges-00000000d6858714-00000000d6880834-132931268309057574-132931303465952525.xz
30.03.2022  19:24             7.172 FileDataChanges-00000000d6882555-00000000d68aa5cc-132931305146910501-132931342089274549.xz
30.03.2022  20:24             9.544 FileDataChanges-00000000d68abbbc-00000000d68d5105-132931343435333719-132931378892854938.xz
30.03.2022  21:24             5.736 FileDataChanges-00000000d68d5fce-00000000d68f14f9-132931379869654038-132931401221496540.xz
30.03.2022  22:34             1.288 FileDataChanges-00000000d68ff01c-00000000d6905b7b-132931416882201367-132931424764512160.xz
30.03.2022  23:59             1.112 FileDataChanges-00000000d692a867-00000000d694fc90-132931459322833115-132931493203288058.xz

C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileDataChanges>dir | find "31.03.2022"
31.03.2022  05:19               548 FileDataChanges-00000000d696407e-00000000d6967368-132931508532584844-132931511981667229.xz
31.03.2022  06:19               872 FileDataChanges-00000000d6a4a671-00000000d6a64eda-132931701467854805-132931724526128667.xz
31.03.2022  07:19             1.416 FileDataChanges-00000000d6a7047f-00000000d6a9093d-132931737946600776-132931766234113422.xz
31.03.2022  08:24             4.564 FileDataChanges-00000000d6a9757a-00000000d6abe89e-132931774085967185-132931811045570140.xz
31.03.2022  09:29            14.976 FileDataChanges-00000000d6ac0d55-00000000d6aea94e-132931813724704759-132931850202734965.xz
31.03.2022  10:34            23.600 FileDataChanges-00000000d6aeb34f-00000000d6b17adf-132931850646549651-132931889194347201.xz
31.03.2022  11:34            25.632 FileDataChanges-00000000d6b17b8a-00000000d6b430e5-132931889250411972-132931925100286301.xz
31.03.2022  12:39            21.364 FileDataChanges-00000000d6b4323e-00000000d6b737e7-132931925266443076-132931963713645702.xz
31.03.2022  13:42            18.644 FileDataChanges-00000000d6b74495-00000000d6ba69b8-132931964361081838-132932001526105660.xz
31.03.2022  14:44            17.364 FileDataChanges-00000000d6ba80c9-00000000d6be4d21-132932002724466278-132932039249775830.xz
31.03.2022  15:49            15.988 FileDataChanges-00000000d6be86de-00000000d6c20976-132932040096451977-132932078304296533.xz
31.03.2022  16:49            21.692 FileDataChanges-00000000d6c21e8e-00000000d6c569be-132932078738751633-132932114237578629.xz
31.03.2022  17:51            16.284 FileDataChanges-00000000d6c573c5-00000000d6cb0ae1-132932114918344513-132932151331085132.xz
31.03.2022  19:16             7.876 FileDataChanges-00000000d6cb25a3-00000000d6cd11bb-132932151906829065-132932183106856709.xz
31.03.2022  20:16             1.708 FileDataChanges-00000000d6cea3ff-00000000d6d06cf5-132932204648587280-132932230340696659.xz
31.03.2022  21:21             2.556 FileDataChanges-00000000d6d0f94d-00000000d6d38e70-132932240777231770-132932276119199432.xz

But in DataReads, the number of files is almost identical but they are much larger.

C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileDataReads>dir | find "29.03.2022"
29.03.2022  05:34             2.900 FileDataReads-00000000d60c9961-00000000d60d84b7-132929665057614989-132929676653941863.xz
29.03.2022  06:33             1.620 FileDataReads-00000000d621d7d2-00000000d623857e-132929981313060003-132929997997294276.xz
29.03.2022  07:40             7.732 FileDataReads-00000000d624a2e0-00000000d62809fa-132930018919200082-132930052686768364.xz
29.03.2022  08:45            46.008 FileDataReads-00000000d628524b-00000000d62b0dbd-132930058071557532-132930095522940349.xz
29.03.2022  09:45           222.464 FileDataReads-00000000d62b11be-00000000d62dbfcc-132930095584624800-132930131554631917.xz
29.03.2022  10:50           306.380 FileDataReads-00000000d62dc9ed-00000000d630a0c8-132930131705470474-132930170552024289.xz
29.03.2022  11:50           265.352 FileDataReads-00000000d630a0cd-00000000d6336cdc-132930170558430791-132930206555596990.xz
29.03.2022  12:50           163.076 FileDataReads-00000000d6336ced-00000000d63635dd-132930206559516080-132930242558325434.xz
29.03.2022  13:50           172.968 FileDataReads-00000000d636364c-00000000d638d7d3-132930242572310761-132930278562303786.xz
29.03.2022  14:55           232.820 FileDataReads-00000000d638d7f6-00000000d63c072b-132930278616662977-132930317547187447.xz
29.03.2022  15:55           133.204 FileDataReads-00000000d63c073c-00000000d63ea415-132930317579195159-132930353385265223.xz
29.03.2022  16:56           281.672 FileDataReads-00000000d63ea70e-00000000d6444252-132930353653443748-132930390629105295.xz
29.03.2022  17:58           199.212 FileDataReads-00000000d6444254-00000000d6480259-132930390631449090-132930427809769003.xz
29.03.2022  18:58            30.120 FileDataReads-00000000d64802f8-00000000d64a7ade-132930427863888999-132930463850712688.xz
29.03.2022  20:53             4.984 FileDataReads-00000000d64a7f45-00000000d64ce251-132930464042416440-132930499655385313.xz
29.03.2022  21:58             7.012 FileDataReads-00000000d64f43ce-00000000d6512960-132930535528014729-132930566951930550.xz

C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileDataReads>dir | find "30.03.2022"
30.03.2022  03:47             1.248 FileDataReads-00000000d651d245-00000000d653fa9b-132930574195682577-132930606025722553.xz
30.03.2022  05:44               612 FileDataReads-00000000d6602314-00000000d6602316-132930783720573974-132930783720573974.xz
30.03.2022  06:44             3.692 FileDataReads-00000000d6659c6c-00000000d667f9f2-132930851174435539-132930887023648277.xz
30.03.2022  07:44             5.340 FileDataReads-00000000d667fef2-00000000d66a1987-132930887181663668-132930917464385505.xz
30.03.2022  08:49            64.896 FileDataReads-00000000d66a9497-00000000d66d5b58-132930923777137507-132930962063866865.xz
30.03.2022  09:54           477.960 FileDataReads-00000000d66d5d2b-00000000d670a239-132930962199167659-132931001111441228.xz
30.03.2022  10:54           111.384 FileDataReads-00000000d670a23a-00000000d67335e7-132931001113641371-132931037046277566.xz
30.03.2022  11:54           125.864 FileDataReads-00000000d673369a-00000000d675dba7-132931037144739357-132931073018894461.xz
30.03.2022  12:59           167.052 FileDataReads-00000000d675dd15-00000000d678b24b-132931073151585945-132931112103796288.xz
30.03.2022  14:04           194.120 FileDataReads-00000000d678b2b0-00000000d67b7f02-132931112157461320-132931151118947701.xz
30.03.2022  15:07           180.432 FileDataReads-00000000d67b7f05-00000000d67e99b5-132931151127541976-132931189077029769.xz
30.03.2022  16:09            89.948 FileDataReads-00000000d67e9a2c-00000000d6826854-132931189080947745-132931226123885530.xz
30.03.2022  17:14            68.632 FileDataReads-00000000d6826cb6-00000000d6855208-132931226300210229-132931265146154925.xz
30.03.2022  18:14            46.428 FileDataReads-00000000d68553ee-00000000d687e3dd-132931265254635974-132931301036711106.xz
30.03.2022  19:14            72.512 FileDataReads-00000000d687ed77-00000000d68a5411-132931301498877353-132931336136948868.xz
30.03.2022  20:19            10.116 FileDataReads-00000000d68a6711-00000000d68cf145-132931337686409550-132931372749106508.xz
30.03.2022  21:24             4.472 FileDataReads-00000000d68d5100-00000000d68f6d7d-132931378892542195-132931407330059553.xz
30.03.2022  22:34             1.000 FileDataReads-00000000d68ff020-00000000d6905b85-132931416883295243-132931424774534873.xz
30.03.2022  23:59               872 FileDataReads-00000000d692a86a-00000000d694fc92-132931459327722308-132931493213689424.xz

C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileDataReads>dir | find "31.03.2022"
31.03.2022  01:47             6.564 FileDataReads-00000000d69637d7-00000000d696a0f9-132931508196173048-132931515275689791.xz
31.03.2022  05:19             1.196 FileDataReads-00000000d69af460-00000000d69d9c5b-132931575722326515-132931611574230965.xz
31.03.2022  06:19             1.052 FileDataReads-00000000d6a4a68c-00000000d6a64fb7-132931701474928358-132931724654323204.xz
31.03.2022  07:19             2.204 FileDataReads-00000000d6a703b5-00000000d6a943ae-132931737807599867-132931770480987390.xz
31.03.2022  08:24            14.108 FileDataReads-00000000d6a97575-00000000d6abe914-132931774084560947-132931811130611997.xz
31.03.2022  09:29           192.376 FileDataReads-00000000d6abf911-00000000d6aea9a3-132931811994816827-132931850242545448.xz
31.03.2022  10:34           273.948 FileDataReads-00000000d6aeab40-00000000d6b17b27-132931850309540765-132931889235828445.xz
31.03.2022  11:34           243.780 FileDataReads-00000000d6b17b94-00000000d6b4322e-132931889254469983-132931925250907135.xz
31.03.2022  12:39           169.504 FileDataReads-00000000d6b4323b-00000000d6b74066-132931925263161257-132931964216671152.xz
31.03.2022  13:42           198.692 FileDataReads-00000000d6b740c9-00000000d6ba6e29-132931964267996530-132932001750634983.xz
31.03.2022  14:44           116.764 FileDataReads-00000000d6ba7ba0-00000000d6be761a-132932002345223187-132932039655240051.xz
31.03.2022  15:49         1.722.012 FileDataReads-00000000d6be7c63-00000000d6c21d27-132932039761155774-132932078709224015.xz
31.03.2022  16:49           341.440 FileDataReads-00000000d6c21e7b-00000000d6c56d33-132932078736876580-132932114474997140.xz
31.03.2022  17:51           928.004 FileDataReads-00000000d6c5728c-00000000d6cb14dc-132932114893743464-132932151712443462.xz
31.03.2022  18:56           137.808 FileDataReads-00000000d6cb1bf0-00000000d6cde6a9-132932151786350324-132932190748707945.xz
31.03.2022  19:56            82.400 FileDataReads-00000000d6cde6ac-00000000d6d03a4d-132932190769045224-132932226758930033.xz
31.03.2022  20:56            92.172 FileDataReads-00000000d6d03a5d-00000000d6d2dbee-132932226779322840-132932262769420862.xz
31.03.2022  21:56            77.104 FileDataReads-00000000d6d2dbf9-00000000d6d52ffd-132932262789866481-132932298751200028.xz
31.03.2022  22:56            44.784 FileDataReads-00000000d6d53013-00000000d6d791eb-132932298792852905-132932334756629253.xz

"Value","Count"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryChanges\FileBinaryChanges-00000000d640acae-00000000d64336ba-132930383320048756-132930384658558137.xz","17"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryChanges\FileBinaryChanges-00000000d645f316-00000000d645f338-132930419321316176-132930419321490243.xz","17"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryChanges\FileBinaryChanges-00000000d64995bd-00000000d64995df-132930455322444124-132930455322600210.xz","17"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryReads\FileBinaryReads-00000000d63e0156-00000000d6414a64-132930347769879292-132930383972888843.xz","32"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryReads\FileBinaryReads-00000000d6419da4-00000000d6462a13-132930384113492746-132930420492579310.xz","28"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileBinaryReads\FileBinaryReads-00000000d64630b6-00000000d649d98b-132930420984228810-132930457698140290.xz","19"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileDataChanges\FileDataChanges-00000000d63ed134-00000000d6447237-132930356621884923-132930393370838740.xz","18"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileDataChanges\FileDataChanges-00000000d64486d2-00000000d6482019-132930394573454676-132930429978684584.xz","18"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileDataReads\FileDataReads-00000000d63ea70e-00000000d6444252-132930353653443748-132930390629105295.xz","34"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileDataReads\FileDataReads-00000000d6444254-00000000d6480259-132930390631449090-132930427809769003.xz","30"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherChanges\FileOtherChanges-00000000d63e2b43-00000000d642f227-132930350566220726-132930384555012201.xz","347"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherChanges\FileOtherChanges-00000000d642f228-00000000d6462af3-132930384555012201-132930420622135119.xz","64"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherChanges\FileOtherChanges-00000000d6462b24-00000000d649da8b-132930420679217011-132930457848741916.xz","33"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherReads\FileOtherReads-00000000d63d27b5-00000000d63fb1d6-132930335566174489-132930371563461079.xz","54"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherReads\FileOtherReads-00000000d63fb1db-00000000d645552f-132930371568340174-132930408628778314.xz","66"
"C:\ProgramData\Sophos\Endpoint Defense\Data\Event Journals\SophosED\FileOtherReads\FileOtherReads-00000000d645553c-00000000d649085c-132930408648863603-132930445837701901.xz","57"

Thanks. Looking at:
https://docs.sophos.com/central/References/schemas/index.html?schema=ld_schema

for the table of interest here: sophos_file_journal, it has the subject column:

So we could as a test, limit the query to a specific subject or possibly subjects and keep the time window I suspect. E.g.

SELECT    
    STRFTIME('%Y-%m-%dT%H:%M:%SZ', DATETIME(file_journal.time,'unixepoch')) AS date_time,
    CASE file_journal.eventType
        WHEN 0 THEN 'Created'
        WHEN 1 THEN 'Renamed'
        WHEN 2 THEN 'Deleted'
        WHEN 3 THEN 'Modified'
        WHEN 4 THEN 'HardLink Created'
        WHEN 5 THEN 'Timestamps Modified'
        WHEN 6 THEN 'Permissions Modified'
        WHEN 7 THEN 'Ownership Modified'
        WHEN 8 THEN 'Accessed'
        WHEN 9 THEN 'Binary File Mapped'
    END AS event_type,
    REPLACE(file_journal.pathname, RTRIM(file_journal.pathname, REPLACE(file_journal.pathname, '\', '')), '') AS file_name,
    file_journal.pathname AS file_path,
    file_journal.sophosPID AS sophos_pid
FROM sophos_file_journal AS file_journal
WHERE
    file_journal.subject = 'FileBinaryChanges'
    AND file_journal.pathname LIKE '$$file_path$$'
    AND file_journal.time > $$start_time$$
    AND file_journal.time < $$end_time$$
ORDER BY file_journal.time DESC

Maybe try just all the changes, just the WHERE clause and below:

WHERE
    file_journal.subject in( 'FileBinaryChanges','FileDataChanges','FileOtherChanges')
    AND file_journal.pathname LIKE '$$file_path$$'
    AND file_journal.time > $$start_time$$
    AND file_journal.time < $$end_time$$
ORDER BY file_journal.time DESC

...but I suspect multiple could still be an issue?

You could try 6 queries, one for each and see if any return and which ones do not.  That would tell us something.

You could potentially go back to the original query for the extra decoration and add in the subject filter if you're interested in just FileDataReads for example.  I suppose it depends on what you're looking for.

I heard at some point there might be a change coming to run it at lower priority so the CPU is kept down but would take longer.  This could help prevent the CPU trigger.

Hi Sophos User930,

I really appreciate your support and knowledge here. Thank you very much!! But currently I cannot put more time into this. Tech support is also holding the case at low level "works as designed" #05100415.

For PM of that product I can say:

This product is a "fail" with this limitations*. We're using XDR Advanced + MTR on our servers and putting a lot of coins in your wallet. On the other hand, the product is full of limitations, that you only notice when you need to use it. Nobody would tell you before you buy. It does not work when your machines meet the System specs and the software has no way to work around. This is very disappointing.

Will speak to our Sales rep about the issue and rethink of using XDR Adv. on our servers.

*Limitations:
https://support.sophos.com/support/s/article/KB-000034920?language=en_US

https://support.sophos.com/support/s/article/KB-000039257?language=en_US#FurtherInformation

where at least this is not true, as can be seen in the thread you're reading here.

As expected, it is currently not possible to query file access on file servers with 2 CPU due to strict process termination after 9 seconds when the process SophosOsqueryExtension.exe uses more than 30% CPU.

What do you do when your trying to find out which files have been read / copied on a file server by malware?

The recommendation was to use Data Lake Queries.

As far as I know, file reads are not logged there and there is no default query for that.

I think you will have to reduce the timeframe initially and possibly export to CSV each report, beyond that, maybe bring the  subject column into play when querying the sophos_file_journal table as I assume it will then return with one "subject". I suspect the FileDataReads subject is the problem, as you mention the files are a lot larger.  

Sophos User930 said:

I think you will have to reduce the timeframe initially

yes, that's currently the only "workaround". I can select 2 hours on that server then it works. If I select 3 hours, it get's aborted.

Not a scenario if you need to span a query over the whole XDR Adv timeframe.

That live discover feature needs a way to disable that watchdog. It stops your show and significantly reduces your ROI.