Recurrent Models of Visual Attention

Abstract

Applying convolutional neural networks to large images is computationally ex-pensive because the amount of computation scales linearly with the number of image pixels. We present a novel recurrent neural network model that is ca-pable of extracting information from an image or video by adaptively selecting a sequence of regions or locations and only processing the selected regions at high resolution. Like convolutional neural networks, the proposed model has a degree of translation invariance built-in, but the amount of computation it per-forms can be controlled independently of the input image size. While the model is non-differentiable, it can be trained using reinforcement learning methods to learn task-specific policies. We evaluate our model on several image classification tasks, where it significantly outperforms a convolutional neural network baseline on cluttered images, and on a dynamic visual control problem, where it learns to track a simple object without an explicit training signal for doing so. 1

Keywords

Computer scienceArtificial intelligenceConvolutional neural networkComputationDifferentiable functionPattern recognition (psychology)Task (project management)Image (mathematics)PixelTranslation (biology)Recurrent neural networkArtificial neural networkImage translationComputer visionAlgorithmMathematics

Affiliated Institutions

Related Publications

Attention Is All You Need

Ashish Vaswani , Noam Shazeer , Niki Parmar +5 more

The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration. The best performing models also co...

2025 6466 citations

Is object localization for free? - Weakly-supervised learning with convolutional neural networks

Maxime Oquab , Léon Bottou , Ivan Laptev +1 more

Successful methods for visual object recognition typically rely on training datasets containing lots of richly annotated images. Detailed image annotation, e.g. by object boundi...

2015 915 citations

CutMix: Regularization Strategy to Train Strong Classifiers With Localizable Features

Sangdoo Yun , Dongyoon Han , Sanghyuk Chun +3 more

Regional dropout strategies have been proposed to enhance performance of convolutional neural network classifiers. They have proved to be effective for guiding the model to atte...

2019 4293 citations

Deep neural networks are easily fooled: High confidence predictions for unrecognizable images

Anh‐Tu Nguyen , Jason Yosinski , Jeff Clune

Deep neural networks (DNNs) have recently been achieving state-of-the-art performance on a variety of pattern-recognition tasks, most notably visual classification problems. Giv...

2015 3232 citations

Image Style Transfer Using Convolutional Neural Networks

Leon A. Gatys , Alexander S. Ecker , Matthias Bethge

Rendering the semantic content of an image in different styles is a difficult image processing task. Arguably, a major limiting factor for previous approaches has been the lack ...

2016 5772 citations

Publication Info

Year: 2014
Type: preprint
Citations: 2271
Access: Closed

External Links

Citation Metrics

2271

OpenAlex

Cite This

APA Style

                            
                                    Volodymyr Mnih, 
                                
                                    Nicolas Heess, 
                                
                                    Alex Graves
                                
                                et al.
                            
                            (2014). 
                            Recurrent Models of Visual Attention. 
                            arXiv (Cornell University)
                            
                            .