Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields

Abstract

We present an approach to efficiently detect the 2D pose of multiple people in an image. The approach uses a nonparametric representation, which we refer to as Part Affinity Fields (PAFs), to learn to associate body parts with individuals in the image. The architecture encodes global context, allowing a greedy bottom-up parsing step that maintains high accuracy while achieving realtime performance, irrespective of the number of people in the image. The architecture is designed to jointly learn part locations and their association via two branches of the same sequential prediction process. Our method placed first in the inaugural COCO 2016 keypoints challenge, and significantly exceeds the previous state-of-the-art result on the MPII Multi-Person benchmark, both in performance and efficiency.

Keywords

Benchmark (surveying)Computer scienceContext (archaeology)Representation (politics)Image (mathematics)Artificial intelligenceParsingPoseProcess (computing)ArchitectureGreedy algorithmMachine learningAerial imageComputer visionPattern recognition (psychology)AlgorithmProgramming language

Affiliated Institutions

Carnegie Mellon University US

Related Publications

OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields

Zhe Cao , Gines Hidalgo , Tomas Simon +2 more

Realtime multi-person 2D pose estimation is a key component in enabling machines to have an understanding of people in images and videos. In this work, we present a realtime app...

2019 IEEE Transactions on Pattern Analysis... 4593 citations

Cascaded Pyramid Network for Multi-person Pose Estimation

Yilun Chen , Zhicheng Wang , Yuxiang Peng +3 more

The topic of multi-person pose estimation has been largely improved recently, especially with the development of convolutional neural network. However, there still exist a lot o...

2018 2018 IEEE/CVF Conference on Computer ... 1733 citations

End-to-End Recovery of Human Shape and Pose

Angjoo Kanazawa , Michael J. Black , David W. Jacobs +1 more

We describe Human Mesh Recovery (HMR), an end-to-end framework for reconstructing a full 3D mesh of a human body from a single RGB image. In contrast to most current methods tha...

2018 2018 IEEE/CVF Conference on Computer ... 1811 citations

DeepCut: Joint Subset Partition and Labeling for Multi Person Pose Estimation

Leonid Pishchulin , Eldar Insafutdinov , Siyu Tang +4 more

This paper considers the task of articulated human pose estimation of multiple people in real world images. We propose an approach that jointly solves the tasks of detection and...

2016 1069 citations

Using k-Poselets for Detecting People and Localizing Their Keypoints

Georgia Gkioxari , Bharath Hariharan , Ross Girshick +1 more

A k-poselet is a deformable part model (DPM) with k parts, where each of the parts is a poselet, aligned to a specific configuration of keypoints based on ground-truth annotatio...

2014 172 citations

Publication Info

Year: 2017
Type: preprint
Pages: 1302-1310
Citations: 7012
Access: Closed

External Links

View on DOI.org

Social Impact

Altmetric

Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields

PlumX Metrics

Social media, news, blog, policy document mentions

Citation Metrics

7012

OpenAlex

Cite This

APA Style

                            
                                    Zhe Cao, 
                                
                                    Tomas Simon, 
                                
                                    Shih-En Wei
                                
                                et al.
                            
                            (2017). 
                            Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields. 
                            
                            , 1302-1310.
                            https://doi.org/10.1109/cvpr.2017.143

Identifiers

DOI: 10.1109/cvpr.2017.143