Learning Spatiotemporal Features with 3D Convolutional Networks

Abstract

We propose a simple, yet effective approach for spatiotemporal feature learning using deep 3-dimensional convolutional networks (3D ConvNets) trained on a large scale supervised video dataset. Our findings are three-fold: 1) 3D ConvNets are more suitable for spatiotemporal feature learning compared to 2D ConvNets, 2) A homogeneous architecture with small 3x3x3 convolution kernels in all layers is among the best performing architectures for 3D ConvNets, and 3) Our learned features, namely C3D (Convolutional 3D), with a simple linear classifier outperform state-of-the-art methods on 4 different benchmarks and are comparable with current best methods on the other 2 benchmarks. In addition, the features are compact: achieving 52.8% accuracy on UCF101 dataset with only 10 dimensions and also very efficient to compute due to the fast inference of ConvNets. Finally, they are conceptually very simple and easy to train and use.

Keywords

Computer scienceArtificial intelligenceConvolutional neural networkClassifier (UML)Pattern recognition (psychology)Convolution (computer science)InferenceDeep learningFeature (linguistics)Feature extractionHomogeneousMachine learningArtificial neural networkMathematics

Affiliated Institutions

Related Publications

Learning Deconvolution Network for Semantic Segmentation

Hyeonwoo Noh , Seunghoon Hong , Bohyung Han

We propose a novel semantic segmentation algorithm by learning a deep deconvolution network. We learn the network on top of the convolutional layers adopted from VGG 16-layer ne...

2015 3978 citations

Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation

Ross Girshick , Jeff Donahue , Trevor Darrell +1 more

Object detection performance, as measured on the canonical PASCAL VOC dataset, has plateaued in the last few years. The best-performing methods are complex ensemble systems that...

2014 30615 citations

Emerging Properties in Self-Supervised Vision Transformers

Mathilde Caron , Hugo Touvron , Ishan Misra +4 more

In this paper, we question if self-supervised learning provides new properties to Vision Transformer (ViT) that stand out compared to convolutional networks (convnets). Beyond t...

2021 2021 IEEE/CVF International Conferenc... 4220 citations

GhostNet: More Features From Cheap Operations

Kai Han , Yunhe Wang , Qi Tian +3 more

Deploying convolutional neural networks (CNNs) on embedded devices is difficult due to the limited memory and computation resources. The redundancy in feature maps is an importa...

2020 3894 citations

Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation

Liang-Chieh Chen , Yukun Zhu , George Papandreou +2 more

Spatial pyramid pooling module or encode-decoder structure are used in deep neural networks for semantic segmentation task. The former networks are able to encode multi-scale co...

2018 Lecture notes in computer science 13300 citations

Publication Info

Year: 2015
Type: preprint
Pages: 4489-4497
Citations: 9293
Access: Closed

External Links

View on DOI.org

Social Impact

Altmetric

Learning Spatiotemporal Features with 3D Convolutional Networks

PlumX Metrics

Social media, news, blog, policy document mentions

Citation Metrics

9293

OpenAlex

Cite This

APA Style

                            
                                    Du Tran, 
                                
                                    Lubomir Bourdev, 
                                
                                    Rob Fergus
                                
                                et al.
                            
                            (2015). 
                            Learning Spatiotemporal Features with 3D Convolutional Networks. 
                            
                            , 4489-4497.
                            https://doi.org/10.1109/iccv.2015.510

Identifiers

DOI: 10.1109/iccv.2015.510