Group Optimization Using Item Response Theory for Peer Assessment

Thien Duc NGUYEN  Masaki UTO  Maomi UENO  

D - Abstracts of IEICE TRANSACTIONS on Information and Systems (Japanese Edition)   Vol.J101-D   No.2   pp.431-445
Publication Date: 2018/02/01
Online ISSN: 1881-0225
Type of Manuscript: PAPER
peer assessment,  item response theory,  group formation,  rater selection,  ability measurement,  optimization problem,  

Full Text(in Japanese): PDF(1.1MB)
>>Buy this Article

As an assessment method based on social constructivism, peer assessment has attracted much attention in recent years. When learners increase as in MOOCs, peer assessment is often conducted by dividing learners into groups. However, in this case, the accuracy of peer assessment depends on a way of forming groups. To optimize the accuracy, this study develops a group optimization method using item response theory. However, experimental results show that the method cannot sufficiently improve the accuracy compared to random groups. Therefore, the study further proposes an external rater selection method to assign a few appropriate outside-group raters to each learner. Experimental results demonstrate that the proposed method can sufficiently improve the accuracy.