Active Learning with Model Selection -- Simultaneous Optimization of Sample Points and Models for Trigonometric Polynomial Models

Masashi SUGIYAMA  Hidemitsu OGAWA  

Publication
IEICE TRANSACTIONS on Information and Systems   Vol.E86-D   No.12   pp.2753-2763
Publication Date: 2003/12/01
Online ISSN: 
DOI: 
Print ISSN: 0916-8532
Type of Manuscript: PAPER
Category: Pattern Recognition
Keyword: 
supervised learning,  generalization capability,  active learning,  model selection,  trigonometric polynomial space,  

Full Text: PDF(966.8KB)>>
Buy this Article




Summary: 
In supervised learning, the selection of sample points and models is crucial for acquiring a higher level of the generalization capability. So far, the problems of active learning and model selection have been independently studied. If sample points and models are simultaneously optimized, then a higher level of the generalization capability is expected. We call this problem active learning with model selection. However, active learning with model selection can not be generally solved by simply combining existing active learning and model selection techniques because of the active learning/model selection dilemma: the model should be fixed for selecting sample points and conversely the sample points should be fixed for selecting models. In this paper, we show that the dilemma can be dissolved if there is a set of sample points that is optimal for all models in consideration. Based on this idea, we give a practical procedure for active learning with model selection in trigonometric polynomial models. The effectiveness of the proposed procedure is demonstrated through computer simulations.