Abstract
Statistical learning theory was introduced in the late 1960's. Until the 1990's it was a purely theoretical analysis of the problem of function estimation from a given collection of data. In the middle of the 1990's new types of learning algorithms (called support vector machines) based on the developed theory were proposed. This made statistical learning theory not only a tool for the theoretical analysis but also a tool for creating practical algorithms for estimating multidimensional functions. This article presents a very general overview of statistical learning theory including both theoretical and algorithmic aspects of the theory. The goal of this overview is to demonstrate how the abstract learning theory established conditions for generalization which are more general than those discussed in classical statistical paradigms and how the understanding of these conditions inspired new algorithmic approaches to function estimation problems. A more detailed overview of the theory (without proofs) can be found in Vapnik (1995). In Vapnik (1998) one can find detailed description of the theory (including proofs).
Keywords
Statistical learning theoryComputer scienceAlgorithmic learning theoryStatistical theoryArtificial intelligenceGeneralizationMachine learningComputational learning theoryLearning theoryFunction (biology)Unsupervised learningProbably approximately correct learningInformation theoryTheoretical computer scienceSupport vector machineMathematics
Affiliated Institutions
Related Publications
Structural risk minimization over data-dependent hierarchies
The paper introduces some generalizations of Vapnik's (1982) method of structural risk minimization (SRM). As well as making explicit some of the details on SRM, it provides a r...
Advances in kernel methods: support vector learning
Introduction to support vector learning roadmap. Part 1 Theory: three remarks on the support vector method of function estimation, Vladimir Vapnik generalization performance of ...
An Introduction to Computational Learning Theory
Emphasizing issues of computational efficiency, Michael Kearns and Umesh Vazirani introduce a number of central topics in computational learning theory for researchers and stude...
Generalization performance of regularization networks and support vector machines via entropy numbers of compact operators
We derive new bounds for the generalization error of kernel machines, such as support vector machines and related regularization networks by obtaining new bounds on their coveri...
Statistical Learning Theory
A comprehensive look at learning and generalization theory. The statistical theory of learning and generalization concerns the problem of choosing desired functions on the basis...
Publication Info
- Year
- 1999
- Type
- article
- Volume
- 10
- Issue
- 5
- Pages
- 988-999
- Citations
- 6095
- Access
- Closed
External Links
Social Impact
Altmetric
PlumX Metrics
Social media, news, blog, policy document mentions
Citation Metrics
6095
OpenAlex
Cite This
Vladimir Vapnik
(1999).
An overview of statistical learning theory.
IEEE Transactions on Neural Networks
, 10
(5)
, 988-999.
https://doi.org/10.1109/72.788640
Identifiers
- DOI
- 10.1109/72.788640