Monocular Object Instance Segmentation and Depth Ordering with CNNs

Abstract

In this paper we tackle the problem of instance-level segmentation and depth ordering from a single monocular image. Towards this goal, we take advantage of convolutional neural nets and train them to directly predict instance-level segmentations where the instance ID encodes the depth ordering within image patches. To provide a coherent single explanation of an image we develop a Markov random field which takes as input the predictions of convolutional neural nets applied at overlapping patches of different resolutions, as well as the output of a connected component algorithm. It aims to predict accurate instance-level segmentation and depth ordering. We demonstrate the effectiveness of our approach on the challenging KITTI benchmark and show good performance on both tasks.

Keywords

MonocularArtificial intelligenceBenchmark (surveying)Markov random fieldConvolutional neural networkComputer scienceSegmentationPattern recognition (psychology)Image segmentationImage (mathematics)Object (grammar)Computer visionGeography

Affiliated Institutions

University of Toronto CA

Related Publications

FCOS: Fully Convolutional One-Stage Object Detection

Zhi Tian , Chunhua Shen , Hao Chen +1 more

We propose a fully convolutional one-stage object detector (FCOS) to solve object detection in a per-pixel prediction fashion, analogue to semantic segmentation. Almost all stat...

2019 5672 citations

UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation

Zongwei Zhou , Md Mahfuzur Rahman Siddiquee , Nima Tajbakhsh +1 more

The state-of-the-art models for medical image segmentation are variants of U-Net and fully convolutional networks (FCN). Despite their success, these models have two limitations...

2019 IEEE Transactions on Medical Imaging 3567 citations

Deep convolutional neural fields for depth estimation from a single image

Fayao Liu , Chunhua Shen , Guosheng Lin

We consider the problem of depth estimation from a sin- gle monocular image in this work. It is a challenging task as no reliable depth cues are available, e.g., stereo corre- s...

2015 863 citations

Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-scale Convolutional Architecture

David Eigen , Rob Fergus

In this paper we address three different computer vision tasks using a single basic architecture: depth prediction, surface normal estimation, and semantic labeling. We use a mu...

2015 2834 citations

CutMix: Regularization Strategy to Train Strong Classifiers With Localizable Features

Sangdoo Yun , Dongyoon Han , Sanghyuk Chun +3 more

Regional dropout strategies have been proposed to enhance performance of convolutional neural network classifiers. They have proved to be effective for guiding the model to atte...

2019 4293 citations

Publication Info

Year: 2015
Type: article
Citations: 140
Access: Closed

External Links

View on DOI.org

Social Impact

Altmetric

Monocular Object Instance Segmentation and Depth Ordering with CNNs

PlumX Metrics

Social media, news, blog, policy document mentions

Citation Metrics

140

OpenAlex

Cite This

APA Style

                            
                                    Ziyu Zhang, 
                                
                                    Alexander G. Schwing, 
                                
                                    Sanja Fidler
                                
                                et al.
                            
                            (2015). 
                            Monocular Object Instance Segmentation and Depth Ordering with CNNs. 
                            
                            .
                            https://doi.org/10.1109/iccv.2015.300

Identifiers

DOI: 10.1109/iccv.2015.300