This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

KnowBe4 RanSim - CryptoGuard didn't stop InsideCryptor

I recently run KnowBe4's ransomware simulator on my desktop. Whilst it appears that Sophos did better than most of their competitors it still missed InsideCryptor. I'm looking for an explanation on how this could happen. Is it a configuration problem? An issue with the simulator or did CryptoGuard drop the ball and Sophos is working on this? 



This thread was automatically locked due to age.
Parents
  • I ran the same test today and found similar issues. We have endpoint protection and Intercept X. 

    The tool is called Ransim from KnowBe4 and simulates 16 types of ransomware. I'd be interested if this is a problem with the Ransim tool or if InterceptX isn't seeing them>

     

     

     

  • Hi Simeon,

    A quick way to see if Sophos worked or not is to check to see if the files are encrypted or not. The path is on the right hand column.

  • Hi Simeon,

    I would also advise raising a support case so that our Labs team can take a closer look at this tool and the results you are being provided with.

    Regards,


    Florentino
    Director, Global Community & Digital Support

    Are you a Sophos Partner? | Product Documentation@SophosSupport | Sign up for SMS Alerts
    If a post solves your question, please use the 'Verify Answer' button.
    The Award-winning Home of Sophos Support Videos! - Visit Sophos Techvids
  • Do you see any isolation done by heartbeat with the Sophos Cloud or XG when you run this?  Just want to make sure there are no adverse affects when testing it out.  Otherwise I will try this afternoon.

     

    Respectfully, 

     

    Badrobot

     

  • It is interesting because I got completely different results, as to what was successful.

     

    I think I will open a case or reach out to someone.

    Respectfully, 

     

    Badrobot

     

  • I also wanted to add that I have gone through the endpoint policies in Sophos Central, every option is on or checked except for device isolation which would not have stopped this on the workstation level anyway, prevented it from accessing network resources maybe but not the workstation.  I have the full Advanced Threat and Intercept X licensing as well.

    Respectfully, 

     

    Badrobot

     

  • Hi everyone,

    The RanSim tool is not a realistic test, while some of what it includes does match real world threats they have also invented some new techniques that create misleading and inconsistent results (as demonstrated by the comments above) and importnatly aren't seen in actual malware. Most AV vendors create detections specifically for the RanSim tool even if what they are doing wouldn't even be considered malicious in the real world, this makes the results from this tool relatively pointless as it doesn't accurately reflect how good a security product is against real world threats.

    Testing AV products is a good idea, however you may find it a lot easier if you just read the reports from the independent 3rd parties that actually create the testing standards and are internationally accepted by the Security industry.

    Our CTO published an article on this recently: https://news.sophos.com/en-us/2018/10/23/fair-rigorous-transparent-and-collaborative-cybersecurity-product-testing-is-good-for-customers-and-good-for-the-industry/

    Or you can skip to some recent 3rd party results that tested Sophos alongside other leading AV vendors: https://news.sophos.com/en-us/2019/07/01/more-fantastic-test-results-for-sophos-intercept-x/

    Hope this helps.

  • I would like to add that I agree with PeterM, after running my own tests while the simulator was running I found that the simulator did not encrypt any files for the tests it claimed I was vulnerable to but also required administrator privileges to even run many aspects of the test which in many cases the average users does not even have. Sophos also flagged all 16 attacks in Sophos Central which again does not add up to the report I was getting.  I did reach out to the reps at knowb4 regarding these results as well, thinking maybe this is something they were not aware of and asking  if it is possible that they are false positives and or have they proven their app against Sophos security apps and the response I got was-

     

    We have a support document that covers issues with false positives (towards the end), should be what you're looking for https://support.knowbe4.com/hc/en-us/articles/229040167-RanSim

    Respectfully, 

     

    Badrobot

     

  • Interesting thread overall. I did run the tests on a Windows 10 1903 using the latest version of RanSim and found the same results as badrobot, namely Injector, ReflectiveInjector and VirlockVariant are not caught by the Sophos InterceptX endpoint.

    I checked the test result folders and the files did get encrypted, so whether those are realistic tests or not, damage can be done using those techniques.

    According to KnowBe4 (see support.knowbe4.com/.../229040167-RanSim ,the techniques used are similar to what could be found in known Ransomware.

    I'd like to get some clarification, for my benefit and hopefully for the rest of the community as well:

    1) PeterM, are you saying that the encryption techniques used in the test, while potent (they did encrypt files), are not realistic in the sense that there are no documented malware using those techniques is this way?

    2) Then are we to understand that malware like Gandcrab, himera, Rokku and Virlock are properly caught by Sophos on the basis of other behavior/traits?

    3) I would assume that as far as encrypting a series of files, there must be behavior/traits that differentiate malware from a person legitimately using some code to encrypt their own files for protection, as the *intention* of the person running a particular piece of code remains unknown to computer programs (at least so far ;-) ).


    I guess my confusion (and I think I'm not the only one) comes from the fact that it is no clear *in what respect* PeterM's argument is valid and sufficient for us to be assured that the Sophos engine does indeed protects from the real world equivalent of those failed tests.

    Thank you
    Christian

  • I have to apologize I believe I looked in the wrong folder the first time, the files for the 3 tests did get encrypted, not everything was reported in Central either, it was on the computer in terms of notifications or prompts as I was running the tests but not in the events tab on Sophos on the endpoint or in Central.  I also had a little more time this afternoon to run more efficient tests so to speak, the pains of managing multiple IT support aspects again I do apologize.

    So as stated above the test results from the Reflective Injector, Virlock Variant and Injector show as vulnerable and did have the files encrypted.  What confuses me a bit is that as I was watching the tests run and I did run the tests three times to ensure no mistakes is the prompts Sophos Central throws up. 

    At one point the prompt shows-

    Which shows the attack as seen as Generic Malware however this would have been the attack tested as the Virlock Variant. In this case the file has already been encrypted when the prompt shows, the attack also shows as Mal/Generic-S, which according to Sophos is "Mal/Generic-S is a name used by Sophos products when detecting a threat via the cloud using Sophos Live Protection." And is not listed as Ransomware.   

    MalGeneric-S Found Here: https://www.sophos.com/en-us/threat-center/threat-analyses/viruses-and-spyware/Mal~Generic-S/detailed-analysis.aspx

    Now on the tests it did block and had no encrypted files in the folders Sophos would throw up a prompt such as “Ransomware Blocked” and the associated folder did not have encrypted files within, which is great because that is what should happen right?

    Now add to this while I have been working on this reply and checking on things, I have noticed that Sophos is still working even though the tests have completed on the RamSim application.  For example, a few minutes after the application completed.  In which RanSim showed test folders 1, 10 & 14 as vulnerable and the files were encrypted.  Sophos was able to revert 3 files in both 10 & 14 back to CSV files.  When you look in the Sophos app and the events tab on the computer you can also see this as-

    HPmal/RanSim-B partially cleaned up.

    Now here is where things just don't add up, I ran the tests or overall RanSim application 3 times, I received 18 notification emails and 18 alerts in Sophos Central, all of which were detected as Ransomware, in each case it is the same 6 aspects that are detected. 

    There is no notification for the Mal/Generic-S which is odd because I see Sophos flag it as I showed in the above screen shot and did also see more than 6 pop up prompts during each test.  This really does not add up since RanSim runs a total of 18 tests so I should be seeing 54 emails and 54 alerts in Sophos Central.  That is unless some of these tests are being blocked by updates or patched software and as such Sophos does not have to do anything anyway.  

    So that is about it, I am going to dive into the Threat Analysis Center after this to see what I can learn about a Ransomware attack, but I wanted to add the above so I had a more accurate description of the testing I have seen between RanSim and Sophos.

    A few other notes that I want to mention based on responses from RanSim and or Sophos-

    I did see mention of hardware playing a factor so I would like to add that I am running a i7-6500U processor with 12GB of RAM and an SSD, I was not over utilizing any resources when I ran the test.

    Sophos did respond to my support case as well on this, some of this would answer these questions so I felt I should add it to this post-

    Regarding the Virlock test. This is an exploit that we've protected against since 2013. However, KnowBe4 does a variation of this exploit that's more of a simulation than an actual exploit. Since nothing is actually being exploited in Windows Intercept X is
    not triggered. However, we are working on detecting KnowBe4's specific test, and this will be released in an upcoming release.

     Regarding the Injector and ReflectiveInjector vulnerabilities. The description appears to simply be an exploit know a Process Hollowing, which is another type of exploit we do protect against and have for a while. While I've not seen that particular vulnerability
    flag, after speaking to some of my colleagues in our Global Escalations team, we do believe it to be similar to the Virlock exploit test.

     

    I know Sophos and Knowbe4 have both been a part of this debate and each side is weighing their opinion and I do appreciate the concerns from all parties regarding this and want to thank everyone for helping with this forum post! 

    If possible I am wondering 2 things-

    One aspect I would like to know is one, is it possible to run these tests individually?  And another is why is the reporting the on-device level not matching with the management system or Central? i.e. Why am I not seeing all the attempts in Sophos Central?

    Respectfully, 

     

    Badrobot

     

Reply
  • I have to apologize I believe I looked in the wrong folder the first time, the files for the 3 tests did get encrypted, not everything was reported in Central either, it was on the computer in terms of notifications or prompts as I was running the tests but not in the events tab on Sophos on the endpoint or in Central.  I also had a little more time this afternoon to run more efficient tests so to speak, the pains of managing multiple IT support aspects again I do apologize.

    So as stated above the test results from the Reflective Injector, Virlock Variant and Injector show as vulnerable and did have the files encrypted.  What confuses me a bit is that as I was watching the tests run and I did run the tests three times to ensure no mistakes is the prompts Sophos Central throws up. 

    At one point the prompt shows-

    Which shows the attack as seen as Generic Malware however this would have been the attack tested as the Virlock Variant. In this case the file has already been encrypted when the prompt shows, the attack also shows as Mal/Generic-S, which according to Sophos is "Mal/Generic-S is a name used by Sophos products when detecting a threat via the cloud using Sophos Live Protection." And is not listed as Ransomware.   

    MalGeneric-S Found Here: https://www.sophos.com/en-us/threat-center/threat-analyses/viruses-and-spyware/Mal~Generic-S/detailed-analysis.aspx

    Now on the tests it did block and had no encrypted files in the folders Sophos would throw up a prompt such as “Ransomware Blocked” and the associated folder did not have encrypted files within, which is great because that is what should happen right?

    Now add to this while I have been working on this reply and checking on things, I have noticed that Sophos is still working even though the tests have completed on the RamSim application.  For example, a few minutes after the application completed.  In which RanSim showed test folders 1, 10 & 14 as vulnerable and the files were encrypted.  Sophos was able to revert 3 files in both 10 & 14 back to CSV files.  When you look in the Sophos app and the events tab on the computer you can also see this as-

    HPmal/RanSim-B partially cleaned up.

    Now here is where things just don't add up, I ran the tests or overall RanSim application 3 times, I received 18 notification emails and 18 alerts in Sophos Central, all of which were detected as Ransomware, in each case it is the same 6 aspects that are detected. 

    There is no notification for the Mal/Generic-S which is odd because I see Sophos flag it as I showed in the above screen shot and did also see more than 6 pop up prompts during each test.  This really does not add up since RanSim runs a total of 18 tests so I should be seeing 54 emails and 54 alerts in Sophos Central.  That is unless some of these tests are being blocked by updates or patched software and as such Sophos does not have to do anything anyway.  

    So that is about it, I am going to dive into the Threat Analysis Center after this to see what I can learn about a Ransomware attack, but I wanted to add the above so I had a more accurate description of the testing I have seen between RanSim and Sophos.

    A few other notes that I want to mention based on responses from RanSim and or Sophos-

    I did see mention of hardware playing a factor so I would like to add that I am running a i7-6500U processor with 12GB of RAM and an SSD, I was not over utilizing any resources when I ran the test.

    Sophos did respond to my support case as well on this, some of this would answer these questions so I felt I should add it to this post-

    Regarding the Virlock test. This is an exploit that we've protected against since 2013. However, KnowBe4 does a variation of this exploit that's more of a simulation than an actual exploit. Since nothing is actually being exploited in Windows Intercept X is
    not triggered. However, we are working on detecting KnowBe4's specific test, and this will be released in an upcoming release.

     Regarding the Injector and ReflectiveInjector vulnerabilities. The description appears to simply be an exploit know a Process Hollowing, which is another type of exploit we do protect against and have for a while. While I've not seen that particular vulnerability
    flag, after speaking to some of my colleagues in our Global Escalations team, we do believe it to be similar to the Virlock exploit test.

     

    I know Sophos and Knowbe4 have both been a part of this debate and each side is weighing their opinion and I do appreciate the concerns from all parties regarding this and want to thank everyone for helping with this forum post! 

    If possible I am wondering 2 things-

    One aspect I would like to know is one, is it possible to run these tests individually?  And another is why is the reporting the on-device level not matching with the management system or Central? i.e. Why am I not seeing all the attempts in Sophos Central?

    Respectfully, 

     

    Badrobot

     

Children
No Data