Abstract
We derive a new self-organizing learning algorithm that maximizes the information transferred in a network of nonlinear units. The algorithm does not assume any knowledge of the input distributions, and is defined here for the zero-noise limit. Under these conditions, information maximization has extra properties not found in the linear case (Linsker 1989). The nonlinearities in the transfer function are able to pick up higher-order moments of the input distributions and perform something akin to true redundancy reduction between units in the output representation. This enables the network to separate statistically independent components in the inputs: a higher-order generalization of principal components analysis. We apply the network to the source separation (or cocktail party) problem, successfully separating unknown mixtures of up to 10 speakers. We also show that a variant on the network architecture is able to perform blind deconvolution (cancellation of unknown echoes and reverberation in a speech signal). Finally, we derive dependencies of information transfer on time delays. We suggest that information maximization provides a unifying framework for problems in "blind" signal processing.
Keywords
Blind signal separationMaximizationBlind deconvolutionDeconvolutionComputer scienceAlgorithmRedundancy (engineering)Nonlinear systemGeneralizationArtificial neural networkMathematicsArtificial intelligenceMathematical optimizationChannel (broadcasting)
Affiliated Institutions
Related Publications
Learning Deconvolution Network for Semantic Segmentation
We propose a novel semantic segmentation algorithm by learning a deep deconvolution network. We learn the network on top of the convolutional layers adopted from VGG 16-layer ne...
Image and video upscaling from local self-examples
We propose a new high-quality and efficient single-image upscaling technique that extends existing example-based super-resolution frameworks. In our approach we do not rely on a...
Network In Network
Abstract: We propose a novel deep network structure called In Network (NIN) to enhance model discriminability for local patches within the receptive field. The conventional con...
Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy
Feature selection is an important problem for pattern classification systems. We study how to select good features according to the maximal statistical dependency criterion base...
Computer ‘‘experiment’’ for nonlinear thermodynamics of Couette flow
Nonequilibrium computer simulations reveal that the equation of state of fluids undergoing shear flow, varies with strain rate. This observation prompted the development of a no...
Publication Info
- Year
- 1995
- Type
- article
- Volume
- 7
- Issue
- 6
- Pages
- 1129-1159
- Citations
- 9075
- Access
- Closed
External Links
Social Impact
Altmetric
PlumX Metrics
Social media, news, blog, policy document mentions
Citation Metrics
9075
OpenAlex
Cite This
Anthony J. Bell,
Terrence J. Sejnowski
(1995).
An Information-Maximization Approach to Blind Separation and Blind Deconvolution.
Neural Computation
, 7
(6)
, 1129-1159.
https://doi.org/10.1162/neco.1995.7.6.1129
Identifiers
- DOI
- 10.1162/neco.1995.7.6.1129