Abstract
The transmission of information through a nonlinear noisy neuron has been computed with the following results. The mutual information between input and output signals is, in the large-noise limit, rigorously given by the mean-squared variance of the fluctuations of the output of the nonlinear neuron. The changes of synaptic strengths that tend to maximize the mutual information are qualitatively similar to those obtained by Hebbian learning of the nonlinear neuron. If noise is added homogeneously to all inputs, its strength becomes multiplied by the number of synapses leading, even in the presence of weak noise, to a ``noise breakdown,'' for which all synaptic strengths tend to the same value during learning.
Keywords
PhysicsHebbian theoryNonlinear systemNoise (video)Statistical physicsInformation transferLimit (mathematics)Learning ruleMutual informationInformation transmissionBiological systemArtificial neural networkArtificial intelligenceMathematical analysisComputer scienceQuantum mechanicsMathematicsTelecommunications
Affiliated Institutions
Related Publications
Local Synaptic Learning Rules Suffice to Maximize Mutual Information in a Linear Network
A network that develops to maximize the mutual information between its output and the signal portion of its input (which is admixed with noise) is useful for extracting salient ...
Nonlinear neurons in the low-noise limit: a factorial code maximizes information transfer
We investigate the consequences of maximizing information transfer in a simple neural network (one input layer, one output layer), focusing on the case of nonlinear transfer fun...
Tag-Trigger-Consolidation: A Model of Early and Late Long-Term-Potentiation and Depression
Changes in synaptic efficacies need to be long-lasting in order to serve as a substrate for memory. Experimentally, synaptic plasticity exhibits phases covering the induction of...
The uptake inhibitor L-trans-PDC enhances responses to glutamate but fails to alter the kinetics of excitatory synaptic currents in the hippocampus
1. We have used patch-clamp recording techniques to study the physiological properties of a recently described glutamate uptake blocker, L-trans-pyrrolidine-2,4-dicarboxylic aci...
A Proposed Function for Hippocampal Theta Rhythm: Separate Phases of Encoding and Retrieval Enhance Reversal of Prior Learning
The theta rhythm appears in the rat hippocampal electroencephalogram during exploration and shows phase locking to stimulus acquisition. Lesions that block theta rhythm impair p...
Publication Info
- Year
- 1992
- Type
- article
- Volume
- 46
- Issue
- 4
- Pages
- 2131-2138
- Citations
- 10
- Access
- Closed
External Links
Social Impact
Altmetric
PlumX Metrics
Social media, news, blog, policy document mentions
Citation Metrics
10
OpenAlex
Cite This
Heinz Georg Schuster
(1992).
Learning by maximizing the information transfer through nonlinear noisy neurons and ‘‘noise breakdown’’.
Physical Review A
, 46
(4)
, 2131-2138.
https://doi.org/10.1103/physreva.46.2131
Identifiers
- DOI
- 10.1103/physreva.46.2131