Abstract
As a modification of the notion of algorithmic complexity, the stochastic complexity of a string of data, relative to a class of probabilistic models, is defined to be the fewest number of binary digits with which the data can be encoded by taking advantage of the selected models. The computation of the stochastic complexity produces a model, which may be taken to incorporate all the statistical information in the data that can be extracted with the chosen model class. This model, for example, allows for optimal prediction, and its parameters are optimized both in their values and their number. A fundamental theorem is proved which gives a lower bound for the code length and, therefore, for prediction errors as well. Finally, the notions of "prior information" and the "useful information" in the data are defined in a new way, and a related construct gives a universal test statistic for hypothesis testing.
Keywords
MathematicsProbabilistic logicClass (philosophy)Test statisticUpper and lower boundsConstruct (python library)Statistical modelComputationStatisticAlgorithmStatistical hypothesis testingTheoretical computer scienceComputer scienceStatisticsArtificial intelligence
Related Publications
Bootstrap Methods in Econometrics: Theory and Numerical Performance
The bootstrap is a method for estimating the distribution of an estimator or test statistic by resampling one's data. It amounts to treating the data as if they were the populat...
Bayes Factors
Abstract In a 1935 paper and in his book Theory of Probability, Jeffreys developed a methodology for quantifying the evidence in favor of a scientific theory. The centerpiece wa...
Quantum theory, the Church–Turing principle and the universal quantum computer
It is argued that underlying the Church–Turing hypothesis there is an implicit physical assertion. Here, this assertion is presented explicitly as a physical principle: ‘every f...
On the Power of Quantum Computation
The quantum model of computation is a model, analogous to the probabilistic Turing machine (PTM), in which the normal laws of chance are replaced by those obeyed by particles on...
Multiple Hypothesis Testing
SUMMARY The concept of multiple comparisons between a set of competing hypotheses is examined. A statistic is developed for testing a maintained hypothesis against a series of s...
Publication Info
- Year
- 1986
- Type
- article
- Volume
- 14
- Issue
- 3
- Citations
- 943
- Access
- Closed
External Links
Social Impact
Altmetric
PlumX Metrics
Social media, news, blog, policy document mentions
Citation Metrics
943
OpenAlex
72
Influential
626
CrossRef
Cite This
J. Rissanen
(1986).
Stochastic Complexity and Modeling.
The Annals of Statistics
, 14
(3)
.
https://doi.org/10.1214/aos/1176350051
Identifiers
- DOI
- 10.1214/aos/1176350051