Friday, April 19, 2013

MASTIFF 0.6.0 Released!

The latest version of MASTIFF, 0.6.0, has just been released! Run over to the download site and grab the latest version!

The official changelog is located here, but the major improvements are described below.

Upgrading MASTIFF to the latest version is easy. You can follow this process:
  1. Download and install pydeep.
  2. Download MASTIFF 0.6.0 and untar it.
  3. Run "make test" to ensure you are not missing any dependencies.
  4. Run "sudo make install" to install the latest version.
  5. Copy the analysis plug-ins (the plugins directory in the tarball) to your location of choice and ensure the config file is pointing to that directory.
  6. Add any new options to your MASTIFF config file. The easiest way may be to use sdiff.

Queue

MASTIFF now has a queueing system so multiple files can be analyzed by the framework. To utilize this, give MASTIFF a directory instead of a file to analyze. It will find all files in that directory and its subdirectories, add them to the queue, and begin processing.

The queue is maintained within the MASTIFF database. So, if you have to stop MASTIFF in the middle of its run, it will begin re-processing the queue when its restarted. Some additional options have been added to allow you to work with the queue:
  • --clear-queue: This will clear the current queue.
  • --ignore-queue: This will ignore the queue and just process the file you give it.
Analysis plug-ins are also taking advantage of the queue. The pdf-parser and ZipExtract plug-ins have a new option ("feedback") which allow you to feed files from the plug-ins back into the queue for processing. For example, the ZipExtract plug-in will add all files that were extracted from the archive into the queue for processing.

Fuzzy Hashing

Fuzzy hashing is not something new within MASTIFF. However, we have changed the Python library used for it. Previously, we used pyssdeep but found that there were a number of stability issues with it on OSX and when processing large amounts of files.

Therefore, we have switched to pydeep (https://github.com/kbandla/pydeep). Our testing has shown it to be much more stable thus far.

libmagic

There was some confusion on which Python libmagic libraries to use when installing MASTIFF. To help alleviate some of that, the framework has been modified to use two different libmagic libraries:
If either library is installed, MASTIFF will utilize them.

Other Changes

A number of other bug fixes and improvements have been made. Please see the changelog file for a complete list.

As always, if you have any questions, please email mastiff-project@korelogic.com.

We have alot of great things coming down the pipe for MASTIFF, but if you have any suggestions, enhancements or plug-ins, let us know!