Minimax bounds for sparse PCA with noisy high-dimensional data

Abstract

We study the problem of estimating the leading eigenvectors of a high-dimensional population covariance matrix based on independent Gaussian observations. We establish a lower bound on the minimax risk of estimators under the <i>l</i><sub>2</sub> loss, in the joint limit as dimension and sample size increase to infinity, under various models of sparsity for the population eigenvectors. The lower bound on the risk points to the existence of different regimes of sparsity of the eigenvectors. We also propose a new method for estimating the eigenvectors by a two-stage coordinate selection scheme.

Keywords

MathematicsMinimaxEigenvalues and eigenvectorsEstimatorUpper and lower boundsApplied mathematicsCovariance matrixGaussianDimension (graph theory)PopulationSample size determinationLimit (mathematics)Mathematical optimizationCovarianceStatisticsCombinatoricsMathematical analysis

Affiliated Institutions

Related Publications

Gaussian regression and optimal finite dimensional linear models

Huaiyu Zhu , Christopher K. I. Williams , Richard Rohwer +1 more

The problem of regression under Gaussian assumptions is treated generally. The relationship between Bayesian prediction, regularization and smoothing is elucidated. The ideal re...

1997 Aston Publications Explorer (Aston Un... 84 citations

Testing Substitution Models Within a Phylogenetic Tree

Günter Weiß

Phylogenetic tree reconstruction frequently assumes the homogeneity of the substitution process over the whole tree. To test this assumption statistically, we propose a test bas...

2003 Molecular Biology and Evolution 25 citations

Confidence Sets for the Mean of a Multivariate Normal Distribution

C. Stein

SUMMARY An attempt is made to determine confidence sets for the mean of a multivariate normal distribution with known covariance matrix that take advantage of the fact that the ...

1962 Journal of the Royal Statistical Soci... 249 citations

Derivation of gauge invariance from high-energy unitarity bounds on the<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"><mml:mi>S</mml:mi></mml:math>matrix

John M. Cornwall , David N. Levin , George Tiktopoulos

A systematic search is made for all renormalizable theories of heavy vector bosons. It is argued that in any renormalizable Lagrangian theory high-energy unitarity bounds should...

1974 Physical review. D. Particles, fields... 845 citations

Finite-Dimensional Approximation of Gaussian Processes

Giancarlo Ferrari‐Trecate , Christopher K. I. Williams , Manfred Opper

Gaussian process (GP) prediction suffers from O(n3) scaling with the data set size n. By using a finite-dimensional basis to approximate the GP predictor, the computational comp...

1998 Neural Information Processing Systems 33 citations

Publication Info

Year: 2013
Type: article
Volume: 41
Issue: 3
Pages: 1055-1084
Citations: 182
Access: Closed

External Links

View on DOI.org

Social Impact

Altmetric

Minimax bounds for sparse PCA with noisy high-dimensional data

PlumX Metrics

Social media, news, blog, policy document mentions

Citation Metrics

182

OpenAlex

Cite This

APA Style

                            
                                
                                    Aharon Birnbaum, 
                                
                                    Iain M. Johnstone, 
                                
                                    Boaz Nadler
                                
                                et al.
                            
                            (2013). 
                            Minimax bounds for sparse PCA with noisy high-dimensional data. 
                            The Annals of Statistics
                            , 41
                            (3)
                            , 1055-1084.
                            https://doi.org/10.1214/12-aos1014
                        

Identifiers

DOI: 10.1214/12-aos1014