This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

Live Discover: Query Cancelled: E Process SophosOsqueryExtension.exe exceeded 30% CPU limit

Hi,

I need this Live Response quickly, unfortunately Sophos Intercept X is aborting the Query.

What is this and how do I get to my data? I just want to use that product with a default query!

2022-03-31T14:29:22.937Z [ 9644: 8204] E Process SophosOsqueryExtension.exe exceeded 30% CPU limit for over: 9 counts

2022-03-31T14:29:22.946Z [ 9644: 6484] W Query Cancelled

2022-03-31T14:28:37.916Z [ 9644: 8204] I Starting FileProcessor: C:\ProgramData\Sophos\Live Query\Queries\Incoming
2022-03-31T14:29:12.534Z [ 9644: 6484] I Running LiveQuery: correlationId:29652b93-474f-41a7-8531-c7104b733871 requestJson:{"name":"File access history","query":"SELECT    \n    STRFTIME('%Y-%m-%dT%H:%M:%SZ', DATETIME(file_journal.time,'unixepoch')) AS date_time,\n    process_journal.processName AS process_name,\n    CASE file_journal.eventType\n        WHEN 0 THEN 'Created'\n        WHEN 1 THEN 'Renamed'\n        WHEN 2 THEN 'Deleted'\n        WHEN 3 THEN 'Modified'\n        WHEN 4 THEN 'HardLink Created'\n        WHEN 5 THEN 'Timestamps Modified'\n        WHEN 6 THEN 'Permissions Modified'\n        WHEN 7 THEN 'Ownership Modified'\n        WHEN 8 THEN 'Accessed'\n        WHEN 9 THEN 'Binary File Mapped'\n    END AS event_type,\n    REPLACE(file_journal.pathname, RTRIM(file_journal.pathname, REPLACE(file_journal.pathname, '\\', '')), '') AS file_name,\n    process_journal.pathname AS process_path,\n    file_journal.pathname AS file_path,\n    file_journal.sophosPID AS sophos_pid,\n    process_journal.sha256 AS sha256,\n    process_properties.mlScore AS ml_score,\n    process_properties.puaScore AS pua_score,\n    process_properties.localRep AS local_rep,\n    process_properties.globalRep AS global_rep\nFROM sophos_file_journal AS file_journal\nLEFT JOIN sophos_process_journal AS process_journal\n    ON process_journal.sophosPID = file_journal.sophosPID\n    AND process_journal.time = REPLACE(file_journal.sophosPID, RTRIM(file_journal.sophosPID, REPLACE(file_journal.sophosPID  , ':', '')), '') / 10000000 - 11644473600\nLEFT JOIN sophos_process_properties AS process_properties \n    USING (sophosPID)\nWHERE\n    file_journal.pathname LIKE 'F:\\Folder\\Folder\\Folder%'\n    AND file_journal.time > 1648563081\n    AND file_journal.time < 1648735200\nORDER BY file_journal.time DESC","type":"sophos.mgt.action.RunLiveQuery"}
2022-03-31T14:29:22.937Z [ 9644: 8204] E Process SophosOsqueryExtension.exe exceeded 30% CPU limit for over: 9 counts
2022-03-31T14:29:22.943Z [ 9644: 8204] I Stopping FileProcessor: C:\ProgramData\Sophos\Live Query\Queries\Incoming
2022-03-31T14:29:22.943Z [ 9644: 8204] I Stopping process SophosOsquery.exe
2022-03-31T14:29:22.946Z [ 9644: 6484] W Query Cancelled



This thread was automatically locked due to age.
Parents
  • the VM machine has 2 vCPU - sure - when that process runs, it consumes 50% - what the heck is that 30% limitation? Are you serious, live discover will only run longer than 9 seconds on machines with 4 core CPU??

  • What happens if you change the query to just:

    SELECT    
        STRFTIME('%Y-%m-%dT%H:%M:%SZ', DATETIME(file_journal.time,'unixepoch')) AS date_time,
        CASE file_journal.eventType
            WHEN 0 THEN 'Created'
            WHEN 1 THEN 'Renamed'
            WHEN 2 THEN 'Deleted'
            WHEN 3 THEN 'Modified'
            WHEN 4 THEN 'HardLink Created'
            WHEN 5 THEN 'Timestamps Modified'
            WHEN 6 THEN 'Permissions Modified'
            WHEN 7 THEN 'Ownership Modified'
            WHEN 8 THEN 'Accessed'
            WHEN 9 THEN 'Binary File Mapped'
        END AS event_type,
        REPLACE(file_journal.pathname, RTRIM(file_journal.pathname, REPLACE(file_journal.pathname, '\', '')), '') AS file_name,
        file_journal.pathname AS file_path,
        file_journal.sophosPID AS sophos_pid
    FROM sophos_file_journal AS file_journal
    WHERE
        file_journal.pathname LIKE '$$file_path$$'
        AND file_journal.time > $$start_time$$
        AND file_journal.time < $$end_time$$
    ORDER BY file_journal.time DESC

    This just uses the same time frame as you define in the variables but only reads from the sophos_file_journal table.

    It might be worth running Process Explorer on the client, with the Performance Graph tab of the SophosOsqueryExtension.exe process open.

    If that is still slow, then we can look into the data behind this table.

  • thanks for your answer.

    that's the same with your modified query.

    It's just, that the query is stopped after 10 seconds due to the 30% for 9 seconds soft limit.

    then it stops.

    This would only run if I put 2 more CPU core into that machine. And I think that is not a solution.

  • FormerMember
    0 FormerMember in reply to LHerzog

    To prevent users from causing severe performance degradation, the queries have a CPU and RAM limiter that prevents any one query from locking/degrading a system down to unresponsive. 

    Are you looking for things on the machines within the last 24 hours?

Reply
  • FormerMember
    0 FormerMember in reply to LHerzog

    To prevent users from causing severe performance degradation, the queries have a CPU and RAM limiter that prevents any one query from locking/degrading a system down to unresponsive. 

    Are you looking for things on the machines within the last 24 hours?

Children
  • : "To prevent users from causing severe performance degradation, the queries have a CPU and RAM limiter that prevents any one query from locking/degrading a system down to unresponsive."

    A: I think this is a good setting. But from my point of perspective, there must be a way to disable that limitation directly from the query within Central. If you have a critical situation you would not care about performance - you need results, ASAP!

    Put a checkbox to Central Query like "Run query with highest priority. This can cause heavy load on the target machine(s)!"

    The target machine is a file server. So surely, there are lots of journal files as mentioned by  . Someone modified a special directory and we don't know when. We want to determine what happened afterwards but cannot.

    How can I run that query? I just want to start the query at the evening and I don't care for high CPU load on the file server at night.

    Update: Support sent me this KB: https://support.sophos.com/support/s/article/KB-000039257?language=en_US

    I think it's outdated because I have 9 seconds, written is 12 seconds. And they mention other soft limits as well. I wonder if this product is designed to work on servers where you will find proably more changes than on endpoints. Especially when your servers have been attacked successfully.