An Empirical Evaluation of an Unpacking Method Implemented with Dynamic Binary Instrumentation

Hyung Chan KIM  Tatsunori ORII  Katsunari YOSHIOKA  Daisuke INOUE  Jungsuk SONG  Masashi ETO  Junji SHIKATA  Tsutomu MATSUMOTO  Koji NAKAO  

IEICE TRANSACTIONS on Information and Systems   Vol.E94-D   No.9   pp.1778-1791
Publication Date: 2011/09/01
Online ISSN: 1745-1361
DOI: 10.1587/transinf.E94.D.1778
Print ISSN: 0916-8532
Type of Manuscript: PAPER
Category: Information Network
software security,  dynamic binary instrumentation,  unpacking,  malware,  binary analysis,  

Full Text: PDF(951.3KB)>>
Buy this Article

Many malicious programs we encounter these days are armed with their own custom encoding methods (i.e., they are packed) to deter static binary analysis. Thus, the initial step to deal with unknown (possibly malicious) binary samples obtained from malware collecting systems ordinarily involves the unpacking step. In this paper, we focus on empirical experimental evaluations on a generic unpacking method built on a dynamic binary instrumentation (DBI) framework to figure out the applicability of the DBI-based approach. First, we present yet another method of generic binary unpacking extending a conventional unpacking heuristic. Our architecture includes managing shadow states to measure code exposure according to a simple byte state model. Among available platforms, we built an unpacking implementation on PIN DBI framework. Second, we describe evaluation experiments, conducted on wild malware collections, to discuss workability as well as limitations of our tool. Without the prior knowledge of 6029 samples in the collections, we have identified at around 64% of those were analyzable with our DBI-based generic unpacking tool which is configured to operate in fully automatic batch processing. Purging corrupted and unworkable samples in native systems, it was 72%.