MTGAN: Extending Test Case set for Deep Learning Image Classifier

Erhu LIU  Song HUANG  Cheng ZONG  Changyou ZHENG  Yongming YAO  Jing ZHU  Shiqi TANG  Yanqiu WANG  

Publication
IEICE TRANSACTIONS on Information and Systems   Vol.E104-D    No.5    pp.709-722
Publication Date: 2021/05/01
Publicized: 2021/02/05
Online ISSN: 1745-1361
DOI: 10.1587/transinf.2020EDP7162
Type of Manuscript: PAPER
Category: Software Engineering
Keyword: 
test case,  deep learning,  adversarial example,  GAN,  

Full Text: PDF>>
Buy this Article



Summary: 
During the recent several years, deep learning has achieved excellent results in image recognition, voice processing, and other research areas, which has set off a new upsurge of research and application. Internal defects and external malicious attacks may threaten the safe and reliable operation of a deep learning system and even cause unbearable consequences. The technology of testing deep learning systems is still in its infancy. Traditional software testing technology is not applicable to test deep learning systems. In addition, the characteristics of deep learning such as complex application scenarios, the high dimensionality of input data, and poor interpretability of operation logic bring new challenges to the testing work. This paper focuses on the problem of test case generation and points out that adversarial examples can be used as test cases. Then the paper proposes MTGAN which is a framework to generate test cases for deep learning image classifiers based on Generative Adversarial Network. Finally, this paper evaluates the effectiveness of MTGAN.