Utilising OSSEC for malware detection

OSSEC is a host based intrusion detection system which collects information via agent deployment, agentless deployments (primarily SSH based), or simply forwarding syslog events to the OSSEC server. It is simple, it works, and best of all it is free (GPLv2). One of the features OSSEC provides is 'syscheck', which is used for monitoring the integrity of the system through monitoring, either periodically or real-time, directories, specific files, and registry keys. To do this, syscheck obtains a checksum, or contents depending on the type of check, of the target objects. My focus is purely on the FIM side of syscheck. The output of syscheck is a file, per machine, consisting of SHA1 and MD5 checksums against each file. Whenever the content changes and syscheck runs, a new checksum will be generated and potentially alerted upon. Additionally, OSSEC has a rootcheck process to identify if your systems present symptoms of a rootkit being installed, some malware, configuration items, etc... This, again, is excellent in identifying if the systems have been compromised, but focuses on known signatures and behaviours at the time of scanning. So, it uses known events and known signatures to identify if an alert should be raised, nothing cutting edge. It maintains signatures, updated irregularly, to identify malware. This means, to detect malware using OSSEC you must be switched on to identify suspicious behaviour (ie. heuristics and event correlation). A quick search for ossec malware detection results in multiple discussions about using suspicious or known behaviours (the process of action) to detect malware. What I want is the ability to check the detected files against malware signatures that are constantly updated. The purpose here is to provide an extra level of detection. This should help identify if that suspicious batch file that appeared and then got deleted was a known malware... providing the security analyst with little time to quickly identify malware that has slipped past their anti-malware software. This will not help you if syscheck has not been configured to monitor the right folders.


Anti-malware products are essential in quick identification of, and protection from malware... However, they are not a be all/end all to identification of malware on computer systems. There are some, loose and generalised, restrictions to anti-malware software due to the manner in which they work:
  1. Only one anti-malware software should be installed on your computer and active at any one time.
    1. To provide real-time protection and on-access scanning, hooks into the kernel are used.
  2. Primarily signature based, resulting in detections only of what is already known.
  3. It can be bypassed easily
  4. Your trust in protecting your systems is placed in a single anti-malware vendor.

The combination of the vendor detection engine and the signature database does not mean absolute detection of all known viruses. Implementing malware detection into OSSEC will allow for verification of files against other vendor databases, hopefully, increasing the accuracy of malware detection.

My approach...Keep It Short and Simple:

To achieve this, we make use of the public API offered by Virus Total. This has a limitation of four requests of any type during any given minute. I strongly recommend paying to use their private API for higher request rates and to support their service. Virus Total provides a free service that analyses files against multiple signature databases and vendor scan engines. However, this will only be using the search function for previously provided files only (file reports).
  1. Proof of concept.
    1. Identify how to detect malware.
    2. Identify complexities in storing relevant information.
    3. Identify external resources required.
    4. Write the code.
    5. Verify we can get what is required.
  2. Determine if the concept works and then merge.
    1. "Translate" the code to C.
    2. Submit the changes for review by OSSEC Developers.

The concept...

  1. Collect the hashes and file names we will be checking.
    1. These are stored in /var/ossec/queue/syscheck/ by default.
  2. Collect the system names for which the file names are attributed.
  3. Check the hashes against VirusTotal's API (3 to 4 per second)
  4. Record hashes we've previously checked
  5. Record if the hash is reported as a potential virus
  6. Create a basic report showing the files, systems, and hashes of potential viruses for further investigation.
  7. Additional runs of the program should not recheck files.
    1. This will drastically reduce the run time.
    2. Will reduce the load on VirusTotal's free service.
    3. Obtaining the report repeatedly for a checksum is not overly beneficial, instead a rescan should be issued. This is out of scope... for now.

And the code

You will need to install DBD::SQLite and VT::API, and register with Virus Total to obtain an API key. The code in the two attached files is a first pass:
  1. To see if my idea is possible
  2. And to see if anything useful can be made of this information

ossec-vt-query - check for potential malware.
ossec-virus-report - create extremely simple report of files and systems listed as potential malware.