An Empirical Evaluation of an Unpacking Method Implemented with Dynamic Binary Instrumentation

Hyung Chan KIM
Tatsunori ORII
Katsunari YOSHIOKA
Daisuke INOUE
Jungsuk SONG
Masashi ETO

IEICE TRANSACTIONS on Information and Systems   Vol.E94-D    No.9    pp.1778-1791
Publication Date: 2011/09/01
Online ISSN: 1745-1361
DOI: 10.1587/transinf.E94.D.1778
Print ISSN: 0916-8532
Type of Manuscript: PAPER
Category: Information Network
software security,  dynamic binary instrumentation,  unpacking,  malware,  binary analysis,  

Full Text: PDF(951.3KB)>>
Buy this Article

Many malicious programs we encounter these days are armed with their own custom encoding methods (i.e., they are packed) to deter static binary analysis. Thus, the initial step to deal with unknown (possibly malicious) binary samples obtained from malware collecting systems ordinarily involves the unpacking step. In this paper, we focus on empirical experimental evaluations on a generic unpacking method built on a dynamic binary instrumentation (DBI) framework to figure out the applicability of the DBI-based approach. First, we present yet another method of generic binary unpacking extending a conventional unpacking heuristic. Our architecture includes managing shadow states to measure code exposure according to a simple byte state model. Among available platforms, we built an unpacking implementation on PIN DBI framework. Second, we describe evaluation experiments, conducted on wild malware collections, to discuss workability as well as limitations of our tool. Without the prior knowledge of 6029 samples in the collections, we have identified at around 64% of those were analyzable with our DBI-based generic unpacking tool which is configured to operate in fully automatic batch processing. Purging corrupted and unworkable samples in native systems, it was 72%.

open access publishing via