Evaluating Deep Learning for Image Classification in Adversarial Environment

Ye PENG  Wentao ZHAO  Wei CAI  Jinshu SU  Biao HAN  Qiang LIU  

IEICE TRANSACTIONS on Information and Systems   Vol.E103-D   No.4   pp.825-837
Publication Date: 2020/04/01
Publicized: 2019/12/23
Online ISSN: 1745-1361
DOI: 10.1587/transinf.2019EDP7188
Type of Manuscript: PAPER
Category: Artificial Intelligence, Data Mining
adversarial environment,  deep learning,  evaluating metrics,  image classification,  security evaluation,  

Full Text: PDF(4.2MB)>>
Buy this Article

Due to the superior performance, deep learning has been widely applied to various applications, including image classification, bioinformatics, and cybersecurity. Nevertheless, the research investigations on deep learning in the adversarial environment are still on their preliminary stage. The emerging adversarial learning methods, e.g., generative adversarial networks, have introduced two vital questions: to what degree the security of deep learning with the presence of adversarial examples is; how to evaluate the performance of deep learning models in adversarial environment, thus, to raise security advice such that the selected application system based on deep learning is resistant to adversarial examples. To see the answers, we leverage image classification as an example application scenario to propose a framework of Evaluating Deep Learning for Image Classification (EDLIC) to conduct comprehensively quantitative analysis. Moreover, we introduce a set of evaluating metrics to measure the performance of different attacking and defensive techniques. After that, we conduct extensive experiments towards the performance of deep learning for image classification under different adversarial environments to validate the scalability of EDLIC. Finally, we give some advice about the selection of deep learning models for image classification based on these comparative results.