Live Discovery Query: How to bulk process a CSV List of SHA256 data

 REVIEWED by Sophos 

When hunting for indicators of compromise it is not uncommon to find a list of things you should be checking.  In the example below I will show how to use variables to select some csv data that is under 5KB in size and then convert the data into a table so we can use it for an automated check.

In this example we will look at a list of SHA256 data published by Sophos on our Git repository related to one of the articles we publish for threat research.

NOTE - If the indicator is deterministic, ie something that if present indicates malware or an adversary, Sophos will already be blocking it with the intercept x protection features.  

Going to https://github.com/sophoslabs/IoCs  I can find a variety of files with threat indicators, most of these are fairly old and again anything in these lists is already going to be blocked if you have the protection policies enabled.  But say you find some long list of SHA256 from some other threat feed location and you want to check to see if your systems have been exposed or not.  It may be that the threat Intel is very fresh and you do not know if Sophos already blocks it or it may be that the data is non-deterministic and only indicative of the threat but not sufficient to convict on by itself. We see lots of IOC's that are non-deterministic these are often things like use of powershell, or psexec or behaviors like setting an auto-run registry key.  In this example we are just going to be addressing a list of SHA256 data, but you can easily convert this query to take a list of other things, like say system tools, command lines, urls, ip-addresses and anything else as long as each element has a separator we can convert it into a table that we can then use with the rest of the query.

With the query below we can simply copy that info into the variable field and hit run.  It will do the work of converting the list to a table we can use in the query and then search the sophos_file_hash_journal for any matches.

Just convert the JOIN to a LEFT JOIN if you want to confirm it is parsing the input correctly.

/************************************************************************\
| Convert the CSV or space seperated list of SHA256 values into a TABLE |
| with each row being its own SHA256 value that we want to look for. |
\************************************************************************/

WITH SHA256_LIST(SHA256, str) AS (
   SELECT '', '$$List of SHA256 data$$' || $$delimiter - CHAR(32) space or CHAR(44) comma$$
   UNION ALL
   SELECT substr(str, 0, instr(str, $$delimiter - CHAR(32) space or CHAR(44) comma$$ )),
      substr(str, instr(str, $$delimiter - CHAR(32) space or CHAR(44) comma$$ )+1)
   FROM SHA256_LIST
   WHERE str!=''
   )

SELECT
   'AN IOC was detected',
   CAST(sl.SHA256 AS TEXT) Target_SHA256,
   datetime(sfhj.time,'unixepoch') Date,
   sfhj.fileID,
   sfhj.fileSize,
   sfhj.sha256
FROM SHA256_LIST sl
   JOIN sophos_file_hash_journal sfhj ON sfhj.sha256 = sl.SHA256
   AND sfhj.time > strftime('%s','now','-1 days')
   AND sl.SHA256 != '';

 ---------------------------------------------------------------------------------------

The RAW table of data I used.

So we have found our list of SHA256's