Abstract
An unsupervised developmental algorithm for linear maps is derived which reduces the pixel-entropy (using the measure introduced in previous work) at every update and thus removes pairwise correlations between pixels. Since the measure of pixel-entropy has only a global minimum the algorithm is guaranteed to converge to the minimum entropy map. Such optimal maps have recently been shown to possess cognitively desirable properties and are likely to be used by the nervous system to organize sensory information. The algorithm derived here turns out to be one proposed by Goodall for pairwise decorrelation. It is biologically plausible since in a neural network implementation it requires only data available locally to a neuron. In training over ensembles of two-dimensional input signals with the same spatial power spectrum as natural scenes, networks develop output neurons with center-surround receptive fields similar to those of ganglion cells in the retina. Some technical issues pertinent to developmental algorithms of this sort, such as “symmetry fixing,” are also discussed.
Keywords
Receptive fieldDecorrelationAlgorithmComputer scienceArtificial neural networkPixelEntropy (arrow of time)Pairwise comparisonSensory systemMathematicsArtificial intelligencePattern recognition (psychology)Neuroscience
Affiliated Institutions
Related Publications
Residual Conv-Deconv Grid Network for Semantic Segmentation
This paper presents GridNet, a new Convolutional Neural Network (CNN)\narchitecture for semantic image segmentation (full scene labelling). Classical\nneural networks are implem...
Network In Network
Abstract: We propose a novel deep network structure called In Network (NIN) to enhance model discriminability for local patches within the receptive field. The conventional con...
Probabilistic framework for the adaptation and comparison of image codes
We apply a Bayesian method for inferring an optimal basis to the problem of finding efficient image codes for natural scenes. The basis functions learned by the algorithm are or...
Dilated Residual Networks
Convolutional networks for image classification progressively reduce resolution until the image is represented by tiny feature maps in which the spatial structure of the scene i...
Understanding Convolution for Semantic Segmentation
Recent advances in deep learning, especially deep convolutional neural networks (CNNs), have led to significant improvement over previous semantic segmentation systems. Here we ...
Publication Info
- Year
- 1993
- Type
- article
- Volume
- 5
- Issue
- 1
- Pages
- 45-60
- Citations
- 89
- Access
- Closed
External Links
Social Impact
Altmetric
PlumX Metrics
Social media, news, blog, policy document mentions
Citation Metrics
89
OpenAlex
Cite This
Joseph J. Atick,
Amanda Redlich
(1993).
Convergent Algorithm for Sensory Receptive Field Development.
Neural Computation
, 5
(1)
, 45-60.
https://doi.org/10.1162/neco.1993.5.1.45
Identifiers
- DOI
- 10.1162/neco.1993.5.1.45