Improving the Accuracy of Least-Squares Probabilistic Classifiers

Makoto YAMADA  Masashi SUGIYAMA  Gordon WICHERN  Jaak SIMM  

Publication
IEICE TRANSACTIONS on Information and Systems   Vol.E94-D   No.6   pp.1337-1340
Publication Date: 2011/06/01
Online ISSN: 1745-1361
DOI: 10.1587/transinf.E94.D.1337
Print ISSN: 0916-8532
Type of Manuscript: LETTER
Category: Pattern Recognition
Keyword: 
least-squares probabilistic classifier,  kernel logistic regression,  density ratio,  PASCAL VOC 2010,  freesound,  

Full Text: PDF(114.6KB)>>
Buy this Article




Summary: 
The least-squares probabilistic classifier (LSPC) is a computationally-efficient alternative to kernel logistic regression. However, to assure its learned probabilities to be non-negative, LSPC involves a post-processing step of rounding up negative parameters to zero, which can unexpectedly influence classification performance. In order to mitigate this problem, we propose a simple alternative scheme that directly rounds up the classifier's negative outputs, not negative parameters. Through extensive experiments including real-world image classification and audio tagging tasks, we demonstrate that the proposed modification significantly improves classification accuracy, while the computational advantage of the original LSPC remains unchanged.