Spectral Normalization for Generative Adversarial Networks

Abstract

One of the challenges in the study of generative adversarial networks is the instability of its training. In this paper, we propose a novel weight normalization technique called spectral normalization to stabilize the training of the discriminator. Our new normalization technique is computationally light and easy to incorporate into existing implementations. We tested the efficacy of spectral normalization on CIFAR10, STL-10, and ILSVRC2012 dataset, and we experimentally confirmed that spectrally normalized GANs (SN-GANs) is capable of generating images of better or equal quality relative to the previous training stabilization techniques.

Keywords

Normalization (sociology)DiscriminatorAdversarial systemComputer scienceArtificial intelligenceGenerative grammarPattern recognition (psychology)Generative adversarial networkImplementationAlgorithmDeep learningDetectorTelecommunications

Affiliated Institutions

Related Publications

High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs

Ting-Chun Wang , Ming-Yu Liu , Jun-Yan Zhu +3 more

We present a new method for synthesizing high-resolution photo-realistic images from semantic label maps using conditional generative adversarial networks (conditional GANs). Co...

2018 4266 citations

Generative Adversarial Networks: An Overview

Antonia Creswell , Tom White , Vincent Dumoulin +3 more

Generative adversarial networks (GANs) provide a way to learn deep representations without extensively annotated training data. They achieve this by deriving backpropagation sig...

2018 IEEE Signal Processing Magazine 4073 citations

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Sergey Ioffe , Christian Szegedy

Training Deep Neural Networks is complicated by the fact that the distribution of each layer's inputs changes during training, as the parameters of the previous layers change. T...

2024 arXiv (Cornell University) 15635 citations

Generative adversarial networks

Ian Goodfellow , Jean Pouget-Abadie , Mehdi Mirza +5 more

Generative adversarial networks are a kind of artificial intelligence algorithm designed to solve the generative modeling problem. The goal of a generative model is to study a c...

2020 Communications of the ACM 11829 citations

Auto-encoder bottleneck features using deep belief networks

Tara N. Sainath , Brian Kingsbury , Bhuvana Ramabhadran

Neural network (NN) bottleneck (BN) features are typically created by training a NN with a middle bottleneck layer. Recently, an alternative structure was proposed which trains ...

2012 181 citations

Publication Info

Year: 2018
Type: article
Citations: 1759
Access: Closed

External Links

Citation Metrics

1759

OpenAlex

Cite This

APA Style

                            
                                    Takeru Miyato, 
                                
                                    Toshiki Kataoka, 
                                
                                    Masanori Koyama
                                
                                et al.
                            
                            (2018). 
                            Spectral Normalization for Generative Adversarial Networks. 
                            arXiv (Cornell University)
                            
                            .