Online Malware Analysis Sandbox Comparison

At the end of January, I decided that I wanted to see if malwr.com, which runs the Cuckoo Sandbox, was as good as other online sandboxes.  This would be important information because the Cuckoo Sandbox can be run locally on your own machines as your own personal sandnet.  If it was proven to be as good as the other online sandboxes, then it would be worth setting up our own Cuckoo farm.

First off, this was an unscientific study done by my analysts with some submissions from other participants on the Emerging Threats mailing list.  With less than one hundred samples analyzed, it's hardly a significant sample size, but it was enough to give us a good representation of what we should expect for our use case.  The files we submitted were collected from drive-by-download exploit kits in the wild.  See the raw data for Trojan family names, MD5's, and other specific information.  Our scoring was as follows:

-1: Technical failure (web site down, no results of any kind returned, etc.)
0: Error when running
1: Results returned, but no network info
2: Results with network info (DNS, HTTP requests)

What we found was that Malwr.com was essentially just as good as Anubis, which means that it's worth running our own local Cuckoo Sandbox for automated analysis which could be fed by Suricata's new file-extract feature, Bro's file-extract feature, or StreamDB object extractions.