Abstract
The exact form of a gradient-following learning algorithm for completely recurrent networks running in continually sampled time is derived and used as the basis for practical algorithms for temporal supervised learning tasks. These algorithms have (1) the advantage that they do not require a precisely defined training interval, operating while the network runs; and (2) the disadvantage that they require nonlocal communication in the network being trained and are computationally expensive. These algorithms allow networks having recurrent connections to learn complex tasks that require the retention of information over time periods having either fixed or indefinite length.
Keywords
Computer scienceArtificial neural networkAlgorithmRecurrent neural networkArtificial intelligenceBasis (linear algebra)Machine learningWake-sleep algorithmMathematicsGeneralization error
Affiliated Institutions
Related Publications
Long Short-Term Memory
Learning to store information over extended time intervals by recurrent backpropagation takes a very long time, mostly because of insufficient, decaying error backflow. We brief...
An application of recurrent nets to phone probability estimation
This paper presents an application of recurrent networks for phone probability estimation in large vocabulary speech recognition. The need for efficient exploitation of context ...
Random-Walk Computation of Similarities between Nodes of a Graph with Application to Collaborative Recommendation
This work presents a new perspective on characterizing the similarity between elements of a database or, more generally, nodes of a weighted and undirected graph. It is based on...
Non-local Neural Networks
Both convolutional and recurrent operations are building blocks that process one local neighborhood at a time. In this paper, we present non-local operations as a generic family...
Generalization of Back propagation to Recurrent and Higher Order Neural Networks
A general method for deriving backpropagation algorithms for networks with recurrent and higher order networks is introduced. The propagation of activation in these networks is ...
Publication Info
- Year
- 1989
- Type
- article
- Volume
- 1
- Issue
- 2
- Pages
- 270-280
- Citations
- 4324
- Access
- Closed
External Links
Social Impact
Altmetric
PlumX Metrics
Social media, news, blog, policy document mentions
Citation Metrics
4324
OpenAlex
Cite This
Ronald J. Williams,
David Zipser
(1989).
A Learning Algorithm for Continually Running Fully Recurrent Neural Networks.
Neural Computation
, 1
(2)
, 270-280.
https://doi.org/10.1162/neco.1989.1.2.270
Identifiers
- DOI
- 10.1162/neco.1989.1.2.270