Abstract
A new method, called the nonnegative (nn) garrote, is proposed for doing subset regression. It both shrinks and zeroes coefficients. In tests on real and simulated data, it produces lower prediction error than ordinary subset selection. It is also compared to ridge regression. If the regression equations generated by a procedure do not change drastically with small changes in the data, the procedure is called stable. Subset selection is unstable, ridge is very stable, and the nn-garrote is intermediate. Simulation results illustrate the effects of instability on prediction error.
Keywords
RegressionRidgeMathematicsSelection (genetic algorithm)Regression analysisStatisticsMean squared prediction errorLinear regressionStepwise regressionStability (learning theory)Applied mathematicsComputer scienceArtificial intelligenceMachine learningGeology
Affiliated Institutions
Related Publications
Better Subset Regression Using the Nonnegative Garrote
Abstract A new method, called the nonnegative (nn) garrote, is proposed for doing subset regression. It both shrinks and zeroes coefficients. In tests on real and simulated data...
The Little Bootstrap and other Methods for Dimensionality Selection in Regression: X-Fixed Prediction Error
Abstract When a regression problem contains many predictor variables, it is rarely wise to try to fit the data by means of a least squares regression on all of the predictor var...
Heuristics of instability and stabilization in model selection
In model selection, usually a "best" predictor is chosen from a collection ${\\hat{\\mu}(\\cdot, s)}$ of predictors where $\\hat{\\mu}(\\cdot, s)$ is the minimum least-squares p...
Simultaneous Regression Shrinkage, Variable Selection, and Supervised Clustering of Predictors with OSCAR
Summary Variable selection can be challenging, particularly in situations with a large number of predictors with possibly high correlations, such as gene expression data. In thi...
A bootstrap resampling procedure for model building: Application to the cox regression model
Abstract A common problem in the statistical analysis of clinical studies is the selection of those variables in the framework of a regression model which might influence the ou...
Publication Info
- Year
- 1995
- Type
- article
- Volume
- 37
- Issue
- 4
- Pages
- 373-373
- Citations
- 785
- Access
- Closed
External Links
Social Impact
Altmetric
PlumX Metrics
Social media, news, blog, policy document mentions
Citation Metrics
785
OpenAlex
Cite This
Leo Breiman
(1995).
Better Subset Regression Using the Nonnegative Garrote.
Technometrics
, 37
(4)
, 373-373.
https://doi.org/10.2307/1269730
Identifiers
- DOI
- 10.2307/1269730