Understanding Developer Commenting in Code Reviews

Toshiki HIRAO  Raula GAIKOVINA KULA  Akinori IHARA  Kenichi MATSUMOTO  

Publication
IEICE TRANSACTIONS on Information and Systems   Vol.E102-D   No.12   pp.2423-2432
Publication Date: 2019/12/01
Publicized: 2019/09/11
Online ISSN: 1745-1361
DOI: 10.1587/transinf.2019MPP0005
Type of Manuscript: Special Section PAPER (Special Section on Empirical Software Engineering)
Category: 
Keyword: 
modern code review,  review comments,  mining software repositories,  empirical study,  machine learning,  

Full Text: PDF(1MB)>>
Buy this Article



Summary: 
Modern code review is a well-known practice to assess the quality of software where developers discuss the quality in a web-based review tool. However, this lightweight approach may risk an inefficient review participation, especially when comments becomes either excessive (i.e., too many) or underwhelming (i.e., too few). In this study, we investigate the phenomena of reviewer commenting. Through a large-scale empirical analysis of over 1.1 million reviews from five OSS systems, we conduct an exploratory study to investigate the frequency, size, and evolution of reviewer commenting. Moreover, we also conduct a modeling study to understand the most important features that potentially drive reviewer comments. Our results find that (i) the number of comments and the number of words in the comments tend to vary among reviews and across studied systems; (ii) reviewers change their behaviours in commenting over time; and (iii) human experience and patch property aspects impact the number of comments and the number of words in the comments.