Abstract
The Marquardt algorithm for nonlinear least squares is presented and is incorporated into the backpropagation algorithm for training feedforward neural networks. The algorithm is tested on several function approximation problems, and is compared with a conjugate gradient algorithm and a variable learning rate algorithm. It is found that the Marquardt algorithm is much more efficient than either of the other techniques when the network contains no more than a few hundred weights.
Keywords
BackpropagationAlgorithmLevenberg–Marquardt algorithmArtificial neural networkRpropConjugate gradient methodComputer scienceFeedforward neural networkArtificial intelligenceFunction (biology)Feed forwardNonlinear systemTime delay neural networkTypes of artificial neural networksEngineering
Related Publications
An adaptive least squares algorithm for the efficient training of artificial neural networks
A novel learning algorithm is developed for the training of multilayer feedforward neural networks, based on a modification of the Marquardt-Levenberg least-squares optimization...
Conjugate gradient algorithm for efficient training of artificial neural networks
A novel approach is presented for the training of multilayer feedforward neural networks, using a conjugate gradient algorithm incorporating an appropriate line search algorithm...
Using random weights to train multilayer networks of hard-limiting units
A gradient descent algorithm suitable for training multilayer feedforward networks of processing units with hard-limiting output functions is presented. The conventional backpro...
A direct adaptive method for faster backpropagation learning: the RPROP algorithm
A learning algorithm for multilayer feedforward networks, RPROP (resilient propagation), is proposed. To overcome the inherent disadvantages of pure gradient-descent, RPROP perf...
Training Recurrent Networks by Evolino
In recent years, gradient-based LSTM recurrent neural networks (RNNs) solved many previously RNN-unlearnable tasks. Sometimes, however, gradient information is of little use for...
Publication Info
- Year
- 1994
- Type
- article
- Volume
- 5
- Issue
- 6
- Pages
- 989-993
- Citations
- 7592
- Access
- Closed
External Links
Social Impact
Altmetric
PlumX Metrics
Social media, news, blog, policy document mentions
Citation Metrics
7592
OpenAlex
144
Influential
5871
CrossRef
Cite This
Martin Hagan,
Mohammad Bagher Menhaj
(1994).
Training feedforward networks with the Marquardt algorithm.
IEEE Transactions on Neural Networks
, 5
(6)
, 989-993.
https://doi.org/10.1109/72.329697
Identifiers
- DOI
- 10.1109/72.329697
- PMID
- 18267874