A Hierarchical Bayesian Approach to Regularization Problems with Multiple Hyperparameters

Ryo TAKEUCHI  Susumu NAKAZAWA  Kazuma KOIZUMI  Takashi MATSUMOTO  

Publication
IEICE TRANSACTIONS on Fundamentals of Electronics, Communications and Computer Sciences   Vol.E83-A   No.8   pp.1641-1650
Publication Date: 2000/08/25
Online ISSN: 
DOI: 
Print ISSN: 0916-8508
Type of Manuscript: Special Section PAPER (Special Section on Digital Signal Processing)
Category: Applications of Signal Processing
Keyword: 
regularizer,  marginal likelihood,  

Full Text: PDF>>
Buy this Article




Summary: 
The Tikhonov regularization theory converts ill-posed inverse problems into well-posed problems by putting penalty on the solution sought. Instead of solving an inverse problem, the regularization theory minimizes a weighted sum of "data error" and "penalty" function, and it has been successfully applied to a variety of problems including tomography, inverse scattering, detection of radiation sources and early vision algorithms. Since the function to be minimized is a weighted sum of functions, one should estimate appropriate weights. This is a problem of hyperparameter estimation and a vast literature exists. Another problem is how one should compare a particular penalty function (regularizer) with another. This is a special class of model comparison problems which are generally difficult. A Hierarchical Bayesian scheme is proposed with multiple hyperparameters in order to cope with data containing subsets which consist of different degree of smoothness. The scheme outperforms the previous scheme with single hyperparameter.