A Novel Approach to Address External Validity Issues in Fault Prediction Using Bandit Algorithms

Teruki HAYAKAWA  Masateru TSUNODA  Koji TODA  Keitaro NAKASAI  Amjed TAHIR  Kwabena Ebo BENNIN  Akito MONDEN  Kenichi MATSUMOTO  

IEICE TRANSACTIONS on Information and Systems   Vol.E104-D   No.2   pp.327-331
Publication Date: 2021/02/01
Publicized: 2020/10/30
Online ISSN: 1745-1361
DOI: 10.1587/transinf.2020EDL8098
Type of Manuscript: LETTER
Category: Software Engineering
defect prediction,  multi-armed bandit,  diversity of datasets,  dynamic model selection,  risk-based testing,  

Full Text: PDF(459.4KB)>>
Buy this Article

Various software fault prediction models have been proposed in the past twenty years. Many studies have compared and evaluated existing prediction approaches in order to identify the most effective ones. However, in most cases, such models and techniques provide varying results, and their outcomes do not result in best possible performance across different datasets. This is mainly due to the diverse nature of software development projects, and therefore, there is a risk that the selected models lead to inconsistent results across multiple datasets. In this work, we propose the use of bandit algorithms in cases where the accuracy of the models are inconsistent across multiple datasets. In the experiment discussed in this work, we used four conventional prediction models, tested on three different dataset, and then selected the best possible model dynamically by applying bandit algorithms. We then compared our results with those obtained using majority voting. As a result, Epsilon-greedy with ϵ=0.3 showed the best or second-best prediction performance compared with using only one prediction model and majority voting. Our results showed that bandit algorithms can provide promising outcomes when used in fault prediction.