Abstract
Dealing with high-dimensional input spaces, like visual input, is a challenging task for reinforcement learning (RL). Neuroevolution (NE), used for continuous RL problems, has to either reduce the problem dimensionality by (1) compressing the representation of the neural network controllers or (2) employing a pre-processor (compressor) that transforms the high-dimensional raw inputs into low-dimensional features. In this paper, we are able to evolve extremely small recurrent neural network (RNN) controllers for a task that previously required networks with over a million weights. The high-dimensional visual input, which the controller would normally receive, is first transformed into a compact feature vector through a deep, max-pooling convolutional neural network (MPCNN). Both the MPCNN preprocessor and the RNN controller are evolved successfully to control a car in the TORCS racing simulator using only visual input. This is the first use of deep learning in the context evolutionary RL.
Keywords
Computer scienceReinforcement learningArtificial intelligenceNeuroevolutionDeep learningRecurrent neural networkConvolutional neural networkContext (archaeology)Feature learningPoolingController (irrigation)Artificial neural networkPattern recognition (psychology)Machine learning
Affiliated Institutions
Related Publications
A State-of-the-Art Survey on Deep Learning Theory and Architectures
In recent years, deep learning has garnered tremendous success in a variety of application domains. This new field of machine learning has been growing rapidly and has been appl...
Asynchronous Methods for Deep Reinforcement Learning
We propose a conceptually simple and lightweight framework for deep reinforcement learning that uses asynchronous gradient descent for optimization of deep neural network contro...
Scene Segmentation with DAG-Recurrent Neural Networks
In this paper, we address the challenging task of scene segmentation. In order to capture the rich contextual dependencies over image regions, we propose Directed Acyclic Graph-...
DiSAN: Directional Self-Attention Network for RNN/CNN-Free Language Understanding
Recurrent neural nets (RNN) and convolutional neural nets (CNN) are widely used on NLP tasks to capture the long-term and local dependencies, respectively. Attention mechanisms ...
Network In Network
Abstract: We propose a novel deep network structure called In Network (NIN) to enhance model discriminability for local patches within the receptive field. The conventional con...
Publication Info
- Year
- 2014
- Type
- article
- Pages
- 541-548
- Citations
- 116
- Access
- Closed
External Links
Social Impact
Altmetric
PlumX Metrics
Social media, news, blog, policy document mentions
Citation Metrics
116
OpenAlex
Cite This
Jan Koutník,
Juergen Schmidhuber,
Faustino Gomez
(2014).
Evolving deep unsupervised convolutional networks for vision-based reinforcement learning.
, 541-548.
https://doi.org/10.1145/2576768.2598358
Identifiers
- DOI
- 10.1145/2576768.2598358