
For FullText PDF, please login, if you are a member of IEICE,
or go to Pay Per View on menu list, if you are a nonmember of IEICE.

A Scaling and NonNegative Garrote in SoftThresholding
Katsuyuki HAGIWARA
Publication
IEICE TRANSACTIONS on Information and Systems
Vol.E100D
No.11
pp.27022710 Publication Date: 2017/11/01
Online ISSN: 17451361
DOI: 10.1587/transinf.2016EDP7365
Type of Manuscript: PAPER Category: Artificial Intelligence, Data Mining Keyword: softthresholding, SURE, nonnegative garrote, scaling, wavelet denoising,
Full Text: PDF>>
Summary:
Softthresholding is a sparse modeling method typically applied to wavelet denoising in statistical signal processing. It is also important in machine learning since it is an essential nature of the wellknown LASSO (Least Absolute Shrinkage and Selection Operator). It is known that softthresholding, thus, LASSO suffers from a problem of dilemma between sparsity and generalization. This is caused by excessive shrinkage at a sparse representation. There are several methods for improving this problem in the field of signal processing and machine learning. In this paper, we considered to extend and analyze a method of scaling of softthresholding estimators. In a setting of nonparametric orthogonal regression problem including discrete wavelet transform, we introduced componentwise and datadependent scaling that is indeed identical to nonnegative garrote. We here considered a case where a parameter value of softthresholding is chosen from absolute values of the least squares estimates, by which the model selection problem reduces to the determination of the number of nonzero coefficient estimates. In this case, we firstly derived a risk and construct SURE (Stein's unbiased risk estimator) that can be used for determining the number of nonzero coefficient estimates. We also analyzed some properties of the risk curve and found that our scaling method with the derived SURE is possible to yield a model with low risk and high sparsity compared to a naive softthresholding method with SURE. This theoretical speculation was verified by a simple numerical experiment of wavelet denoising.

