A Hierarchical Bayesian Approach to Regularization Problems with Multiple Hyperparameters


IEICE TRANSACTIONS on Fundamentals of Electronics, Communications and Computer Sciences   Vol.E83-A   No.8   pp.1641-1650
Publication Date: 2000/08/25
Online ISSN: 
Print ISSN: 0916-8508
Type of Manuscript: Special Section PAPER (Special Section on Digital Signal Processing)
Category: Applications of Signal Processing
regularizer,  marginal likelihood,  

Full Text: PDF>>
Buy this Article

The Tikhonov regularization theory converts ill-posed inverse problems into well-posed problems by putting penalty on the solution sought. Instead of solving an inverse problem, the regularization theory minimizes a weighted sum of "data error" and "penalty" function, and it has been successfully applied to a variety of problems including tomography, inverse scattering, detection of radiation sources and early vision algorithms. Since the function to be minimized is a weighted sum of functions, one should estimate appropriate weights. This is a problem of hyperparameter estimation and a vast literature exists. Another problem is how one should compare a particular penalty function (regularizer) with another. This is a special class of model comparison problems which are generally difficult. A Hierarchical Bayesian scheme is proposed with multiple hyperparameters in order to cope with data containing subsets which consist of different degree of smoothness. The scheme outperforms the previous scheme with single hyperparameter.