Jürgen Schmidhuber

Long Short-Term Memory

Learning to store information over extended time intervals by recurrent backpropagation takes a very long time, mostly because of insufficient, decaying error backflow. We brief...

Training Very Deep Networks

Theoretical and empirical evidence indicates that the depth of neural networks is crucial for their success. However, training becomes more difficult as depth increases, and tra...

Highway Networks

There is plenty of theoretical and empirical evidence that depth of neural networks is a crucial ingredient for their success. However, network training becomes more difficult w...

Learning Factorial Codes by Predictability Minimization

I propose a novel general principle for unsupervised learning of distributed nonredundant internal representations of input patterns. The principle is based on two opposing forc...

Deep learning in neural networks: An overview

Connectionist temporal classification

Many real-world sequence learning tasks require the prediction of sequences of labels from noisy, unsegmented input data. In speech recognition, for example, an acoustic signal ...

Framewise phoneme classification with bidirectional LSTM and other neural network architectures

Unconstrained On-line Handwriting Recognition with Recurrent Neural Networks

In online handwriting recognition the trajectory of the pen is recorded during writing. Although the trajectory provides a compact and complete representation of the written out...