Sophos causing High CPU usage and unexpected server restarts

Hey guys,

 

I was experiencing some odd issues with Sophos on our file server since the weekend, on Monday it was reaching high CPU usage for a second then restarting every 30 minutes, this appears to be from a windows "Bugcheck":

 

Error 

Description:
The computer has rebooted from a bugcheck. The bugcheck was: 0x0000003b (0x00000000c0000005, 0xfffff80145467864, 0xffffd00021ecc980, 0x0000000000000000). A dump was saved in: C:\Windows\MEMORY.DMP. Report Id: 090318-16453-01.

After analysing the memory dump it was apparent that Clean.exe was the cause of these issues (Copied out three seperate memory dumps from three seperate reboots, all were from Clean.exe) so I have uninstalled Sophos for the time being and the issues have ceased. I believe the version was 2.0.2.

 

Is there a hotfix for this issue? Is this a bug that's already known? Is there an update that will resolve these resource and bugcheck issues?

 

These issues occured on a Windows Server 2012 R2 VM running off VMware ESXi, 6.5.0, 7967591 

 

Best Regards,

Jason

  • Hello Jason,

    please open a ticket with Support. Someone must be the first one to encounter a bug Wink - ok, the icon is not nice but the statement is true.Support could tell you if the problem is already knwo. There are no hotfixes, resolved issues are in the release notes for new releases or versions.

    Christian

  • In reply to QC:

    Hi Jason, 

    I echo Christians point, please log a support case and then PM me the ticket number so that I can get the logs looked at ASAP.

    Regards,

    Stephen

  • In reply to StephenMcKay:

    Hi,

     

    I've tried logging a support ticket but get "Access denied" when I try and submit it and I'm 100% certain nobody mans the support@sophos.com mailbox as I've emailed them three times over the week with no reply. Is there any way to get around the "Access denied" thing?

  • In reply to Jason Ryan1:

    Hello Jason,

    where do you get this Access denied? Could you perhaps show a screenshot?

    Christian

  • In reply to QC:

    Hey Christian,

     

    I go to: Open a Support Case -> Enduser & Server -> Endpoint (On Premises) -> Fill out the ticket -> Click Next and then I get the access denied error:

     

    Access Denied

    You don't have permission to access "secure2.sophos.com/.../describe-issue.aspx" on this server.

    Reference #18.c1f31502.1536664259.78b2ca

     

    Best Regards,

    Jason

     

  • In reply to Jason Ryan1:

    Hello Jason,

    hm, Next takes me to to the How can we get in touch with you? page. But I do get a security warning, the certificate's subject is www.sophos.com and there's also mixed content. Didn't complete the final step though.

    I'm not Sophos so I can't say what happens there, it's a smart page that can also time out - wait long enough and next takes you to the product selection. If you just enter a severity and a short description - do you get the get in touch page?

    Christian

  • In reply to QC:

    I have also tested the support form and got through to the form to enter my contact details page? 

    PM me your email address you contacted support on directly and I will go and talk to support for you. 

  • In reply to MarkToshack:

    I got your PM Jason - I have spoken to support who has looked at the ticket logging system that automatically logs calls from emails and there isnt one from you. 

    I have also had a look at your company details but cannot see your name as a contact.

    When you tried the support form were you logged in? 

    It might be worth contacting us via the number in this instance: 

    From UK:
    0844 767 4670 (0844 SOPHOS-0)
    International:
    +44 (0)1235 465818

  • HI Jason,

    Do you have a memory dump you could load into Windbg and run:

    !analyze -v

    Regards,
    Jak

  • In reply to jak:

    Hey Jak,

     

    Please find memory dump below:

     


    Microsoft (R) Windows Debugger Version 10.0.17134.12 X86
    Copyright (c) Microsoft Corporation. All rights reserved.


    Loading Dump File [C:\Users\[REMOVED]\Desktop\MEMORY.DMP]
    Kernel Bitmap Dump File: Kernel address space is available, User address space may not be available.

    Symbol search path is: srv*
    Executable search path is:
    Windows 8.1 Kernel Version 9600 MP (2 procs) Free x64
    Product: Server, suite: TerminalServer DataCenter SingleUserTS
    Built by: 9600.18895.amd64fre.winblue_ltsb.180101-1800
    Machine Name:
    Kernel base = 0xfffff803`8be1f000 PsLoadedModuleList = 0xfffff803`8c0ec6d0
    Debug session time: Mon Sep 3 12:38:38.374 2018 (UTC + 1:00)
    System Uptime: 0 days 0:08:26.092
    Loading Kernel Symbols
    ...............................................................
    ................................................................
    ...................
    Loading User Symbols
    PEB is paged out (Peb.Ldr = 00000000`7fa93018). Type ".hh dbgerr001" for details
    Loading unloaded module list
    .....
    *******************************************************************************
    * *
    * Bugcheck Analysis *
    * *
    *******************************************************************************

    Use !analyze -v to get detailed debugging information.

    BugCheck 3B, {c0000005, fffff8010a9cc864, ffffd0002178c980, 0}

    *** ERROR: Module load completed but symbols could not be loaded for sidfile.sys
    Probably caused by : sidfile.sys ( sidfile+1b864 )

    Followup: MachineOwner
    ---------

    0: kd> !analyze -v
    *******************************************************************************
    * *
    * Bugcheck Analysis *
    * *
    *******************************************************************************

    SYSTEM_SERVICE_EXCEPTION (3b)
    An exception happened while executing a system service routine.
    Arguments:
    Arg1: 00000000c0000005, Exception code that caused the bugcheck
    Arg2: fffff8010a9cc864, Address of the instruction which caused the bugcheck
    Arg3: ffffd0002178c980, Address of the context record for the exception that caused the bugcheck
    Arg4: 0000000000000000, zero.

    Debugging Details:
    ------------------


    KEY_VALUES_STRING: 1


    TIMELINE_ANALYSIS: 1


    DUMP_CLASS: 1

    DUMP_QUALIFIER: 401

    BUILD_VERSION_STRING: 9600.18895.amd64fre.winblue_ltsb.180101-1800

    SYSTEM_MANUFACTURER: VMware, Inc.

    VIRTUAL_MACHINE: VMware

    SYSTEM_PRODUCT_NAME: VMware7,1

    SYSTEM_VERSION: None

    BIOS_VENDOR: VMware, Inc.

    BIOS_VERSION: VMW71.00V.0.B64.1704110547

    BIOS_DATE: 04/11/2017

    BASEBOARD_MANUFACTURER: Intel Corporation

    BASEBOARD_PRODUCT: 440BX Desktop Reference Platform

    BASEBOARD_VERSION: None

    DUMP_TYPE: 1

    BUGCHECK_P1: c0000005

    BUGCHECK_P2: fffff8010a9cc864

    BUGCHECK_P3: ffffd0002178c980

    BUGCHECK_P4: 0

    EXCEPTION_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%p referenced memory at 0x%p. The memory could not be %s.

    FAULTING_IP:
    sidfile+1b864
    fffff801`0a9cc864 f6405002 test byte ptr [rax+50h],2

    CONTEXT: ffffd0002178c980 -- (.cxr 0xffffd0002178c980)
    rax=0000000000000000 rbx=0000000000000065 rcx=ffffe000592c8430
    rdx=ffffd0002178d980 rsi=ffffe0004ec96060 rdi=ffffe000592c8430
    rip=fffff8010a9cc864 rsp=ffffd0002178d3b0 rbp=ffffd0002178d601
    r8=ffffe0004ec96060 r9=0000000000000000 r10=0000000000000000
    r11=0000000000000000 r12=ffffe000592c8500 r13=ffffe0004ec96060
    r14=ffffd0002178d980 r15=ffffd0002178d9f0
    iopl=0 nv up ei pl zr na po nc
    cs=0010 ss=0018 ds=002b es=002b fs=0053 gs=002b efl=00010246
    sidfile+0x1b864:
    fffff801`0a9cc864 f6405002 test byte ptr [rax+50h],2 ds:002b:00000000`00000050=??
    Resetting default scope

    CPU_COUNT: 2

    CPU_MHZ: 8fc

    CPU_VENDOR: GenuineIntel

    CPU_FAMILY: 6

    CPU_MODEL: 4f

    CPU_STEPPING: 0

    CPU_MICROCODE: 6,4f,0,0 (F,M,S,R) SIG: 2000043'00000000 (cache) 2000043'00000000 (init)

    DEFAULT_BUCKET_ID: WIN8_DRIVER_FAULT

    BUGCHECK_STR: 0x3B

    PROCESS_NAME: Clean.exe

    CURRENT_IRQL: 0

    ANALYSIS_SESSION_HOST: [REMOVED]

    ANALYSIS_SESSION_TIME: 09-03-2018 13:08:35.0004

    ANALYSIS_VERSION: 10.0.17134.12 x86fre

    LAST_CONTROL_TRANSFER: from fffff8038c1f3076 to fffff8010a9cc864

    STACK_TEXT:
    ffffd000`2178d3b0 fffff803`8c1f3076 : 00000000`00000065 ffffd000`2178d6f1 ffffe000`4ec96060 ffffd000`2178d9f0 : sidfile+0x1b864
    ffffd000`2178d410 fffff803`8c2b4f1e : ffffc001`b88c3768 ffffc001`b88c3768 ffffc001`d4dd19e0 ffffe000`4ec96030 : nt!IopParseDevice+0xa46
    ffffd000`2178d600 fffff803`8c1ed5c3 : 00000000`00000000 ffffd000`2178d7b8 ffffc001`00000040 ffffe000`4eb8cb00 : nt!ObpLookupObjectName+0x7be
    ffffd000`2178d740 fffff803`8c287a50 : ffffe000`00000001 00000000`0450e708 00000000`0450fdb0 00000000`00000001 : nt!ObOpenObjectByName+0x1e3
    ffffd000`2178d870 fffff803`8bf86113 : ffffe000`535ce880 00000000`779fc3e0 ffffe000`535ce880 00000000`7f960000 : nt!NtQueryAttributesFile+0x140
    ffffd000`2178db00 00007ff8`014a0b2a : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiSystemServiceCopyEnd+0x13
    00000000`0450e6c8 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : 0x00007ff8`014a0b2a


    THREAD_SHA1_HASH_MOD_FUNC: 1c2fcfbfbf67a2232f67fb095fde32563a4ff9ca

    THREAD_SHA1_HASH_MOD_FUNC_OFFSET: b1790a26078c23c22151bb6e4c017a5cc581a98c

    THREAD_SHA1_HASH_MOD: d6bbf15f46028098b426b06fb206b9b74e2e11db

    FOLLOWUP_IP:
    sidfile+1b864
    fffff801`0a9cc864 f6405002 test byte ptr [rax+50h],2

    FAULT_INSTR_CODE: 25040f6

    SYMBOL_STACK_INDEX: 0

    SYMBOL_NAME: sidfile+1b864

    FOLLOWUP_NAME: MachineOwner

    MODULE_NAME: sidfile

    IMAGE_NAME: sidfile.sys

    DEBUG_FLR_IMAGE_TIMESTAMP: 58a31a82

    STACK_COMMAND: .cxr 0xffffd0002178c980 ; kb

    BUCKET_ID_FUNC_OFFSET: 1b864

    FAILURE_BUCKET_ID: 0x3B_sidfile!unknown_function

    BUCKET_ID: 0x3B_sidfile!unknown_function

    PRIMARY_PROBLEM_CLASS: 0x3B_sidfile!unknown_function

    TARGET_TIME: 2018-09-03T11:38:38.000Z

    OSBUILD: 9600

    OSSERVICEPACK: 0

    SERVICEPACK_NUMBER: 0

    OS_REVISION: 0

    SUITE_MASK: 400

    PRODUCT_TYPE: 3

    OSPLATFORM_TYPE: x64

    OSNAME: Windows 8.1

    OSEDITION: Windows 8.1 Server TerminalServer DataCenter SingleUserTS

    OS_LOCALE:

    USER_LCID: 0

    OSBUILD_TIMESTAMP: 2018-01-02 03:56:56

    BUILDDATESTAMP_STR: 180101-1800

    BUILDLAB_STR: winblue_ltsb

    BUILDOSVER_STR: 6.3.9600.18895.amd64fre.winblue_ltsb.180101-1800

    ANALYSIS_SESSION_ELAPSED_TIME: 8d3

    ANALYSIS_SOURCE: KM

    FAILURE_ID_HASH_STRING: km:0x3b_sidfile!unknown_function

    FAILURE_ID_HASH: {daee3b38-e2b8-d17f-fb37-24268cce18fd}

    Followup: MachineOwner
    ---------

  • In reply to Jason Ryan1:

    Hi Jason,

    We have a couple of tickets created that report similar issues; the root cause of these is that there is probably a third party driver that takes issues when we call the Windows API GetFileAttributes on its driver.

    Do you use Varonis on this server? 

    Regards,

    Stephen

  • In reply to StephenMcKay:

    Clean.exe (user mode - can't cause a bugcheck of this nature on its own) has called the standard Windows function NtQueryAttributesFile.  It would be hard if that could cause an issue I suspect sidfile.sys, which is later in the stack has taken exception to something.  I would check with Varonis.

    Regards,
    Jak


  • In reply to StephenMcKay:

    Hey Stephen,

     

    We do, yeah. Is this only an issue with the current release of Sophos? We never had these issues before.

     

    Best Regards,

    Jason

  • In reply to jak:

    Hey Stephen,

     

    Is there any way to exclude Varonis from Sophos, or the other way around to prevent this from happening?

     

  • In reply to Jason Ryan1:

    A note on the ticket i am reviewing says 

    'We have received confirmation that Varonis have released a patch that reportedly fixes the issue.' 

    I am trying to ascertain what the patch is, but you might be able to get info directly from them.

    Stephen