Abstract
I present a new way to parallelize the training of convolutional neural networks across multiple GPUs. The method scales significantly better than all alternatives when applied to modern convolutional neural networks.
Keywords
Computer scienceConvolutional neural networkArtificial intelligenceProgramming languageParallel computing
Related Publications
Instant neural graphics primitives with a multiresolution hash encoding
Neural graphics primitives, parameterized by fully connected neural networks, can be costly to train and evaluate. We reduce this cost with a versatile new input encoding that p...
Large Scale Distributed Deep Networks
Recent work in unsupervised feature learning and deep learning has shown that being able to train large models can dramatically improve performance. In this paper, we consider t...
Improved Adam Optimizer for Deep Neural Networks
Adaptive optimization algorithms, such as Adam and RMSprop, have witnessed better optimization performance than stochastic gradient descent (SGD) in some scenarios. However, rec...
Neural GPUs Learn Algorithms
Abstract: Learning an algorithm from examples is a fundamental problem that has been widely studied. Recently it has been addressed using neural networks, in particular by Neura...
GPU-acceleration for Large-scale Tree Boosting
In this paper, we present a novel massively parallel algorithm for accelerating the decision tree building procedure on GPUs (Graphics Processing Units), which is a crucial step...
Publication Info
- Year
- 2014
- Type
- preprint
- Citations
- 980
- Access
- Closed
External Links
Social Impact
Altmetric
PlumX Metrics
Social media, news, blog, policy document mentions
Citation Metrics
980
OpenAlex
Cite This
Alex Krizhevsky
(2014).
One weird trick for parallelizing convolutional neural networks.
arXiv (Cornell University)
.
https://doi.org/10.48550/arxiv.1404.5997
Identifiers
- DOI
- 10.48550/arxiv.1404.5997