Abstract
Convolutional networks almost always incorporate some form of spatial pooling, and very often it is alpha times alpha max-pooling with alpha=2. Max-pooling act on the hidden layers of the network, reducing their size by an integer multiplicative factor alpha. The amazing by-product of discarding 75% of your data is that you build into the network a degree of invariance with respect to translations and elastic distortions. However, if you simply alternate convolutional layers with max-pooling layers, performance is limited due to the rapid reduction in spatial size, and the disjoint nature of the pooling regions. We have formulated a fractional version of max-pooling where alpha is allowed to take non-integer values. Our version of max-pooling is stochastic as there are lots of different ways of constructing suitable pooling regions. We find that our form of fractional max-pooling reduces overfitting on a variety of datasets: for instance, we improve on the state-of-the art for CIFAR-100 without even using dropout.
Keywords
PoolingOverfittingMultiplicative functionComputer scienceDisjoint setsInteger (computer science)Dropout (neural networks)MathematicsArtificial intelligenceCombinatoricsMachine learningArtificial neural network
Affiliated Institutions
Related Publications
Network In Network
Abstract: We propose a novel deep network structure called In Network (NIN) to enhance model discriminability for local patches within the receptive field. The conventional con...
Striving for Simplicity: The All Convolutional Net
Most modern convolutional neural networks (CNNs) used for object recognition are built using the same principles: Alternating convolution and max-pooling layers followed by a sm...
Applying Convolutional Neural Networks concepts to hybrid NN-HMM model for speech recognition
Convolutional Neural Networks (CNN) have showed success in achieving translation invariance for many image processing tasks. The success is largely attributed to the use of loca...
A Fast Learning Algorithm for Deep Belief Nets
We show how to use “complementary priors” to eliminate the explaining-away effects that make inference difficult in densely connected belief nets that have many hidden layers. U...
A survey on Image Data Augmentation for Deep Learning
Abstract Deep convolutional neural networks have performed remarkably well on many Computer Vision tasks. However, these networks are heavily reliant on big data to avoid overfi...
Publication Info
- Year
- 2014
- Type
- preprint
- Citations
- 335
- Access
- Closed
External Links
Social Impact
Altmetric
PlumX Metrics
Social media, news, blog, policy document mentions
Citation Metrics
335
OpenAlex
Cite This
Benjamin Graham
(2014).
Fractional Max-Pooling.
arXiv (Cornell University)
.
https://doi.org/10.48550/arxiv.1412.6071
Identifiers
- DOI
- 10.48550/arxiv.1412.6071