Performance Comparison of Subjective Assessment Methods for Stereoscopic 3D Video Quality

Taichi KAWANO  Kazuhisa YAMAGISHI  Takanori HAYASHI  

IEICE TRANSACTIONS on Communications   Vol.E97-B   No.4   pp.738-745
Publication Date: 2014/04/01
Online ISSN: 1745-1345
DOI: 10.1587/transcom.E97.B.738
Type of Manuscript: PAPER
Category: Network
ACR,  DCR,  DSQCS,  3D,  2D,  

Full Text: PDF(1MB)>>
Buy this Article

The International Telecommunication Union has standardized many subjective assessment methods for stereoscopic three-dimensional (3D) and 2D video quality. The same methods are used for 3D and 2D videos. The assessment time, stability, and discrimination ability, which means the ability to identify differences in video quality, are important factors in subjective assessment methods. Many studies on these factors have been done for 2D video quality. However, these factors for 3D video quality have not been sufficiently studied. To address this, we conduct subjective quality assessments for 3D and 2D videos using the absolute category rating (ACR), degradation category rating (DCR), and double stimulus continuous quality-scale (DSCQS) methods that are defined in ITU Recommendations. We first investigate the Pearson's correlation coefficients and Spearman's rank correlation coefficients between different pairings of the three methods to clarify which method is most efficient in terms of assessment time. The different pairings of the three methods exhibit high coefficients. These results indicate that the order relation of the mean opinion score (MOS) and the distance between the MOSs for these methods are almost the same. Therefore, for generally investigating the quality characteristics, the ACR method is most efficient because it has the shortest assessment time. Next, we analyze the stability of these subjective assessment methods. We clarify that the confidence intervals (CIs) of the MOSs for 3D video are almost the same as those for 2D video and that the stability of the DCR method is higher than that of the other methods. The DSCQS method has the smallest CIs for high-quality video. Finally, we investigate the discrimination ability of these subjective assessment methods. The results show that the DCR method performs better than the others in terms of the number of paired MOSs with a significant difference for low-quality video. However, we confirm that the DSCQS method performs better than the others for high-quality video.