Efficient Piecewise Training of Deep Structured Models for Semantic Segmentation

Guosheng Lin; Chunhua Shen; Anton van den Hengel; Ian Reid

doi:10.1109/cvpr.2016.348

Abstract

Recent advances in semantic image segmentation have mostly been achieved by training deep convolutional neural networks (CNNs). We show how to improve semantic segmentation through the use of contextual information, specifically, we explore 'patch-patch' context between image regions, and 'patch-background' context. For learning from the patch-patch context, we formulate Conditional Random Fields (CRFs) with CNN-based pairwise potential functions to capture semantic correlations between neighboring patches. Efficient piecewise training of the proposed deep structured model is then applied to avoid repeated expensive CRF inference for back propagation. For capturing the patch-background context, we show that a network design with traditional multi-scale image input and sliding pyramid pooling is effective for improving performance. Our experimental results set new state-of-the-art performance on a number of popular semantic segmentation datasets, including NYUDv2, PASCAL VOC 2012, PASCAL-Context, and SIFT-flow. In particular, we achieve an intersection-overunion score of 78:0 on the challenging PASCAL VOC 2012 dataset.

Keywords

Computer scienceArtificial intelligenceConditional random fieldSegmentationCRFSConvolutional neural networkPascal (unit)PiecewiseInferenceDeep learningPattern recognition (psychology)Image segmentationMachine learningMathematics

Affiliated Institutions

Related Publications

Fully Convolutional Networks for Semantic Segmentation

Evan Shelhamer , Jonathan Long , Trevor Darrell

Convolutional networks are powerful visual models that yield hierarchies of features. We show that convolutional networks by themselves, trained end-to-end, pixels-to-pixels, im...

2016 IEEE Transactions on Pattern Analysis... 10715 citations

Fully convolutional networks for semantic segmentation

Jonathan Long , Evan Shelhamer , Trevor Darrell

Convolutional networks are powerful visual models that yield hierarchies of features. We show that convolutional networks by themselves, trained end-to-end, pixels-to-pixels, ex...

2015 35498 citations

RefineNet: Multi-path Refinement Networks for High-Resolution Semantic Segmentation

Guosheng Lin , Anton Milan , Chunhua Shen +1 more

Recently, very deep convolutional neural networks (CNNs) have shown outstanding performance in object recognition and have also been the first choice for dense classification pr...

2017 3120 citations

DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs

Liang-Chieh Chen , George Papandreou , Iasonas Kokkinos +2 more

In this work we address the task of semantic image segmentation with Deep Learning and make three main contributions that are experimentally shown to have substantial practical ...

2017 IEEE Transactions on Pattern Analysis... 20855 citations

Learning Deconvolution Network for Semantic Segmentation

Hyeonwoo Noh , Seunghoon Hong , Bohyung Han

We propose a novel semantic segmentation algorithm by learning a deep deconvolution network. We learn the network on top of the convolutional layers adopted from VGG 16-layer ne...

2015 3978 citations

Publication Info

Year: 2016
Type: article
Citations: 844
Access: Closed

External Links

View on DOI.org

Social Impact

Altmetric

Efficient Piecewise Training of Deep Structured Models for Semantic Segmentation

PlumX Metrics

Social media, news, blog, policy document mentions

Citation Metrics

844

OpenAlex

Cite This

APA Style

                            
                                    Guosheng Lin, 
                                
                                    Chunhua Shen, 
                                
                                    Anton van den Hengel
                                
                                et al.
                            
                            (2016). 
                            Efficient Piecewise Training of Deep Structured Models for Semantic Segmentation. 
                            
                            .
                            https://doi.org/10.1109/cvpr.2016.348

Identifiers

DOI: 10.1109/cvpr.2016.348