Wednesday, February 29, 2012

Online Malware Analysis Sandbox Comparison

At the end of January, I decided that I wanted to see if malwr.com, which runs the Cuckoo Sandbox, was as good as other online sandboxes.  This would be important information because the Cuckoo Sandbox can be run locally on your own machines as your own personal sandnet.  If it was proven to be as good as the other online sandboxes, then it would be worth setting up our own Cuckoo farm.

First off, this was an unscientific study done by my analysts with some submissions from other participants on the Emerging Threats mailing list.  With less than one hundred samples analyzed, it's hardly a significant sample size, but it was enough to give us a good representation of what we should expect for our use case.  The files we submitted were collected from drive-by-download exploit kits in the wild.  See the raw data for Trojan family names, MD5's, and other specific information.  Our scoring was as follows:

-1: Technical failure (web site down, no results of any kind returned, etc.)
0: Error when running
1: Results returned, but no network info
2: Results with network info (DNS, HTTP requests)


What we found was that Malwr.com was essentially just as good as Anubis, which means that it's worth running our own local Cuckoo Sandbox for automated analysis which could be fed by Suricata's new file-extract feature, Bro's file-extract feature, or StreamDB object extractions.

7 comments:

  1. Thanks for publishing your test results. It's very helpful to other teams running their own sandboxen to see the comparisons.

    ReplyDelete
  2. Many thanks for sharing this.
    Taking a look at your result i am quite sure there was no malware DLL in your samples.
    You should incorporate 20% or 30% of DLL :)

    And you'll see ThreatExpert becoming better, malwr having some hard time.

    And also many sample evade Anubis and are caught by Malwr (terminal server registry check).

    ReplyDelete
  3. a big problem with automated sandboxes
    they do not perform all the functions of a given program,
    it is very simple to hide the malicious part of code that will only execute when a certain button is pressed within a program, an automated sandbox will only execute a program they do not "click all the buttons" they only check for malware behavior on execution and when the program exits or terminates, this is just too easy to bypass
    i can imagine it would take a fair bit of extra work for the sandbox to be able to detect every possible interaction that is possible from user mode, and then to run all these functions and then check for malware activity

    ReplyDelete
  4. If you are considering running a sandbox locally you should take a look at Buster Sandbox Analyzer. This malware analyzer uses Sandboxie (www.sandboxie.com) as framework to run the malware safely, without the need of a virtual machine.

    More information about this great tool here: http://bsa.isoftware.nl/

    ReplyDelete
  5. This comment has been removed by the author.

    ReplyDelete
  6. Have you tried http://www.threattracksecurity.com/resources/sandbox-malware-analysis.aspx?

    ReplyDelete
  7. Another alternative: http://www.hybrid-analysis.com

    Highlights: 180+ Generic Behavior Signatures, Static Analysis (Visual Binary Layout, TrID, SSDEEP, IMPHASH, PDF parsing, VBA extraction from DOC/RTF), Supports 32-bit PE files (DLL/EXE/COM/SCR/PIF, etc.), Annotated Disassembly based on Memory Dumps, Extensive String Extraction

    ReplyDelete