Harmony potentials for joint classification and segmentation

Abstract

Hierarchical conditional random fields have been successfully applied to object segmentation. One reason is their ability to incorporate contextual information at different scales. However, these models do not allow multiple labels to be assigned to a single node. At higher scales in the image, this yields an oversimplified model, since multiple classes can be reasonable expected to appear within one region. This simplified model especially limits the impact that observations at larger scales may have on the CRF model. Neglecting the information at larger scales is undesirable since class-label estimates based on these scales are more reliable than at smaller, noisier scales. To address this problem, we propose a new potential, called harmony potential, which can encode any possible combination of class labels. We propose an effective sampling strategy that renders tractable the underlying optimization problem. Results show that our approach obtains state-of-the-art results on two challenging datasets: Pascal VOC 2009 and MSRC-21.

Keywords

Pascal (unit)ENCODESegmentationComputer scienceArtificial intelligenceConditional random fieldPattern recognition (psychology)Class (philosophy)Machine learning

Affiliated Institutions

Related Publications

Learning Deconvolution Network for Semantic Segmentation

Hyeonwoo Noh , Seunghoon Hong , Bohyung Han

We propose a novel semantic segmentation algorithm by learning a deep deconvolution network. We learn the network on top of the convolutional layers adopted from VGG 16-layer ne...

2015 3978 citations

Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation

Liang-Chieh Chen , Yukun Zhu , George Papandreou +2 more

2018 Lecture notes in computer science 13300 citations

Normalized cuts and image segmentation

Jianbo Shi , Jitendra Malik

We propose a novel approach for solving the perceptual grouping problem in vision. Rather than focusing on local features and their consistencies in the image data, our approach...

2000 IEEE Transactions on Pattern Analysis... 15440 citations

Is object localization for free? - Weakly-supervised learning with convolutional neural networks

Maxime Oquab , Léon Bottou , Ivan Laptev +1 more

Successful methods for visual object recognition typically rely on training datasets containing lots of richly annotated images. Detailed image annotation, e.g. by object boundi...

2015 915 citations

Learning Hierarchical Features for Scene Labeling

Clément Farabet , Camille Couprie , Laurent Najman +1 more

Scene labeling consists of labeling each pixel in an image with the category of the object it belongs to. We propose a method that uses a multiscale convolutional network traine...

2012 IEEE Transactions on Pattern Analysis... 2684 citations

Publication Info

Year: 2010
Type: article
Pages: 3280-3287
Citations: 156
Access: Closed

External Links

View on DOI.org

Social Impact

Altmetric

Harmony potentials for joint classification and segmentation

PlumX Metrics

Social media, news, blog, policy document mentions

Citation Metrics

156

OpenAlex

Cite This

APA Style

                            
                                    Josep M. Gonfaus, 
                                
                                    Xavier Boix, 
                                
                                    Joost van de Weijer
                                
                                et al.
                            
                            (2010). 
                            Harmony potentials for joint classification and segmentation. 
                            
                            , 3280-3287.
                            https://doi.org/10.1109/cvpr.2010.5540048

Identifiers

DOI: 10.1109/cvpr.2010.5540048