Empirical Bayes Estimation for L1 Regularization: A Detailed Analysis in the One-Parameter Lasso Model


IEICE TRANSACTIONS on Fundamentals of Electronics, Communications and Computer Sciences   Vol.E101-A    No.12    pp.2184-2191
Publication Date: 2018/12/01
Online ISSN: 1745-1337
DOI: 10.1587/transfun.E101.A.2184
Type of Manuscript: Special Section PAPER (Special Section on Information Theory and Its Applications)
Category: Machine learning
lasso regression,  empirical Bayes,  Laplace prior,  local variational approximation,  

Full Text: PDF(880.1KB)>>
Buy this Article

Lasso regression based on the L1 regularization is one of the most popular sparse estimation methods. It is often required to set appropriately in advance the regularization parameter that determines the degree of regularization. Although the empirical Bayes approach provides an effective method to estimate the regularization parameter, its solution has yet to be fully investigated in the lasso regression model. In this study, we analyze the empirical Bayes estimator of the one-parameter model of lasso regression and show its uniqueness and its properties. Furthermore, we compare this estimator with that of the variational approximation, and its accuracy is evaluated.