Abstract
Two convergence aspects of the EM algorithm are studied: (i) does the EM algorithm find a local maximum or a stationary value of the (incomplete-data) likelihood function? (ii) does the sequence of parameter estimates generated by EM converge? Several convergence results are obtained under conditions that are applicable to many practical situations. Two useful special cases are: (a) if the unobserved complete-data specification can be described by a curved exponential family with compact parameter space, all the limit points of any EM sequence are stationary points of the likelihood function; (b) if the likelihood function is unimodal and a certain differentiability condition is satisfied, then any EM sequence converges to the unique maximum likelihood estimate. A list of key properties of the algorithm is included.
Keywords
MathematicsSequence (biology)Convergence (economics)Likelihood functionExponential familyLimit (mathematics)Function (biology)Expectation–maximization algorithmAlgorithmMaximum likelihoodDifferentiable functionApplied mathematicsExponential functionWeak convergenceEstimation theoryLimit of a functionStatisticsMathematical analysisComputer science
Related Publications
Variable selection using MM algorithms
Variable selection is fundamental to high-dimensional statistical modeling. Many variable selection techniques may be implemented by maximum penalized likelihood using various p...
Simple and Globally Convergent Methods for Accelerating the Convergence of Any EM Algorithm
Abstract. The expectation‐maximization (EM) algorithm is a popular approach for obtaining maximum likelihood estimates in incomplete data problems because of its simplicity and ...
Quasi-likelihood functions, generalized linear models, and the Gauss—Newton method
To define a likelihood we have to specify the form of distribution of the observations, but to define a quasi-likelihood function we need only specify a relation between the mea...
Line Search Filter Methods for Nonlinear Programming: Motivation and Global Convergence
Line search methods are proposed for nonlinear programming using Fletcher and Leyffer's filter method [Math. Program., 91 (2002), pp. 239--269], which replaces the traditional m...
On the Asymptotics of Constrained $M$-Estimation
Limit theorems for an $M$-estimate constrained to lie in a closed subset of $\\mathbb{R}^d$ are given under two different sets of regularity conditions. A consistent sequence of...
Publication Info
- Year
- 1983
- Type
- article
- Volume
- 11
- Issue
- 1
- Citations
- 3244
- Access
- Closed
External Links
Social Impact
Altmetric
PlumX Metrics
Social media, news, blog, policy document mentions
Citation Metrics
3244
OpenAlex
Cite This
Changbao Wu
(1983).
On the Convergence Properties of the EM Algorithm.
The Annals of Statistics
, 11
(1)
.
https://doi.org/10.1214/aos/1176346060
Identifiers
- DOI
- 10.1214/aos/1176346060