Abstract
Sparse coding is a method for finding a representation of data in which each of the components of the representation is only rarely significantly active. Such a representation is closely related to redundancy reduction and independent component analysis, and has some neurophysiological plausibility. In this article, we show how sparse coding can be used for denoising. Using maximum likelihood estimation of nongaussian variables corrupted by gaussian noise, we show how to apply a soft-thresholding (shrinkage) operator on the components of sparse coding so as to reduce noise. Our method is closely related to the method of wavelet shrinkage, but it has the important benefit over wavelet methods that the representation is determined solely by the statistical properties of the data. The wavelet representation, on the other hand, relies heavily on certain mathematical properties (like self-similarity) that may be only weakly related to the properties of natural data.
Keywords
WaveletPattern recognition (psychology)Sparse approximationShrinkageThresholdingNoise reductionNeural codingMathematicsArtificial intelligenceRepresentation (politics)AlgorithmComputer scienceStatistics
Related Publications
A Reliability Coefficient for Maximum Likelihood Factor Analysis
Maximum likelihood factor analysis provides an effective method for estimation of factor matrices and a useful test statistic in the likelihood ratio for rejection of overly sim...
Image Super-Resolution Via Sparse Representation
This paper presents a new approach to single-image super-resolution, based on sparse signal representation. Research on image statistics suggests that image patches can be well-...
Functional Connectivity: The Principal-Component Analysis of Large (PET) Data Sets
The distributed brain systems associated with performance of a verbal fluency task were identified in a nondirected correlational analysis of neurophysiological data obtained wi...
Phylogeny Estimation and Hypothesis Testing Using Maximum Likelihood
One of the strengths of the maximum likelihood method of phylogenetic estimation is the ease with which hypotheses can be formulated and tested. Maximum likelihood analysis of D...
Spline Smoothing: The Equivalent Variable Kernel Method
The spline smoothing approach to nonparametric regression and curve estimation is considered. It is shown that, in a certain sense, spline smoothing corresponds approximately to...
Publication Info
- Year
- 1999
- Type
- article
- Volume
- 11
- Issue
- 7
- Pages
- 1739-1768
- Citations
- 386
- Access
- Closed
External Links
Social Impact
Altmetric
PlumX Metrics
Social media, news, blog, policy document mentions
Citation Metrics
386
OpenAlex
Cite This
Aapo Hyvärinen
(1999).
Sparse Code Shrinkage: Denoising of Nongaussian Data by Maximum Likelihood Estimation.
Neural Computation
, 11
(7)
, 1739-1768.
https://doi.org/10.1162/089976699300016214
Identifiers
- DOI
- 10.1162/089976699300016214