Studying the Cost and Effectiveness of OSS Quality Assessment Models: An Experience Report of Fujitsu QNET

Yasutaka KAMEI  Takahiro MATSUMOTO  Kazuhiro YAMASHITA  Naoyasu UBAYASHI  Takashi IWASAKI  Shuichi TAKAYAMA  

Publication
IEICE TRANSACTIONS on Information and Systems   Vol.E101-D   No.11   pp.2744-2753
Publication Date: 2018/11/01
Online ISSN: 1745-1361
DOI: 10.1587/transinf.2018EDP7163
Type of Manuscript: PAPER
Category: Software Engineering
Keyword: 
open source software,  OSS quality assessment models,  empirical studies,  applied research,  

Full Text: PDF(600.1KB)
>>Buy this Article


Summary: 
Nowadays, open source software (OSS) systems are adopted by proprietary software projects. To reduce the risk of using problematic OSS systems (e.g., causing system crashes), it is important for proprietary software projects to assess OSS systems in advance. Therefore, OSS quality assessment models are studied to obtain information regarding the quality of OSS systems. Although the OSS quality assessment models are partially validated using a small number of case studies, to the best of our knowledge, there are few studies that empirically report how industrial projects actually use OSS quality assessment models in their own development process. In this study, we empirically evaluate the cost and effectiveness of OSS quality assessment models at Fujitsu Kyushu Network Technologies Limited (Fujitsu QNET). To conduct the empirical study, we collect datasets from (a) 120 OSS projects that Fujitsu QNET's projects actually used and (b) 10 problematic OSS projects that caused major problems in the projects. We find that (1) it takes average and median times of 51 and 49 minutes, respectively, to gather all assessment metrics per OSS project and (2) there is a possibility that we can filter problematic OSS systems by using the threshold derived from a pool of assessment metrics. Fujitsu QNET's developers agree that our results lead to improvements in Fujitsu QNET's OSS assessment process. We believe that our work significantly contributes to the empirical knowledge about applying OSS assessment techniques to industrial projects.