Abstract

We compare the performance of three types of neural network-based ensemble techniques to that of a single neural network. The ensemble algorithms are two versions of boosting and committees of neural networks trained independently. For each of the four algorithms, we experimentally determine the test and training error curves in an optical character recognition (OCR) problem as both a function of training set size and computational cost using three architectures. We show that a single machine is best for small training set size while for large training set size some version of boosting is best. However, for a given computational cost, boosting is always best. Furthermore, we show a surprising result for the original boosting algorithm: namely, that as the training set size increases, the training error decreases until it asymptotes to the test error rate. This has potential implications in the search for better training algorithms.

Keywords

Boosting (machine learning)Artificial neural networkGradient boostingComputer scienceMachine learningArtificial intelligenceAsymptoteWord error rateTraining setTest setEnsemble learningAlgorithmPattern recognition (psychology)Mathematics

Affiliated Institutions

Related Publications

Publication Info

Year
1994
Type
article
Volume
6
Issue
6
Pages
1289-1301
Citations
352
Access
Closed

External Links

View on DOI.org

Social Impact

Altmetric
PlumX Metrics

Social media, news, blog, policy document mentions

Citation Metrics

352
OpenAlex

Cite This

Harris Drucker, Corinna Cortes, L. D. Jackel et al. (1994). Boosting and Other Ensemble Methods. Neural Computation , 6 (6) , 1289-1301. https://doi.org/10.1162/neco.1994.6.6.1289

Identifiers

DOI
10.1162/neco.1994.6.6.1289