Object Recognition, Detection and 6D Pose Estimation

State of the Art Methods and Datasets

Accurate localization and pose estimation of 3D objects is of great importance to many higher level tasks such as robotic manipulation (like Amazon Picking Challenge), scene interpretation and augmented reality to name a few. The recent introduction of consumer-level depth sensors have allowed for substantial improvement over traditional 2D approaches as finer 3D geometrical features can be captured. However, there still remain several challenges to address including:

  • Foreground occlusions
  • Background clutter
  • Large scale and pose changes
  • Multi-instace objects

The scope of this website is to list state of the art methods and datasets available to further help drive research.

State of the Art Methods
Title Year Notes
Pose Guided RGBD Feature Learning for 3D Object Pose Estimation
V. Balntas, A. Doumanoglou, C. Sahin, J. Sock, R. Kouskouridas, T-K Kim
 ICCV 2017 paper
Real-Time Monocular Pose Estimation of 3D Objects Using Temporally Consistent Local Color Histograms
H. Tjaden, U. Schwanecke, E. Schomer
 ICCV 2017 paper – video
Rotational Subgroup Voting and Pose Clustering for Robust 3D Object Recognition
A. G. Buch, L. Kiforenko, D. Kraft
 ICCV 2017 paper
SSD-6D- Making RGB-Based 3D Detection and 6D Pose Estimation Great Again
W. Kehl, F. Manhardt, F. Tombari, S. Ilic, N. Navab
 ICCV 2017 paper – supplementary material
BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting the 3D Poses of Challenging Objects without Using Depth
M. Rad, V. Lepetit
 ICCV 2017 paper – project
BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting the 3D Poses of Challenging Objects without Using Depth
M. Rad, V. Lepetit
 ICCV 2017 paper – project
Global Hypothesis Generation for 6D Object Pose Estimation
F. Michel, A. Kirillov, E. Brachmann, A. Krull, S. Gumhold, B. Savchynskyy, C. Rother
 CVPR 2017 paper
PoseAgent-Budget-Constrained 6D Object Pose Estimation via Reinforcement Learning
A. Krull, E. Brachmann, S. Nowozin, F. Michel, J. Shotton, C. Rother
 CVPR 2017 paper
Going Further with Point Pair Features
S. Hinterstoisser, V. Lepetit, N. Rajkumar, K. Konolige
 ECCV 2016 paper
Deep Learning of Local RGB-D Patches for 3D Object Detection and 6D Pose Estimation
W. Kehl, F. Milletari, F. Tombari, S. Ilic, N. Navab
 ECCV 2016 paper – video
Uncertainty-Driven 6D Pose Estimation of Objects and Scenes from a Single RGB Image
E. Brachmann, F. Michel, A. Krull, M. Y. Yang, S. Gumhold, C. Rother
 CVPR 2016 paper – project page
Recovering 6D Object Pose and Predicting Next-Best-View in the Crowd
A. Doumanoglou, R. Kouskouridas, S. Malassiotis, T-K, Kim
 CVPR 2016 paper – project page – supplementary material
Learning Analysis-by-Synthesis for 6D Pose Estimation in RGB-D Images
A. Krull, E. Brachmann, F. Michel, M. Y. Yang, S. Gumhold, C. Rother
 ICCV 2015 paper – project page – video
A Novel Representation of Parts for Accurate 3D Object Detection and Tracking in Monocular Images
A. Crivellaro, M. Rad, Y. Verdie, K. M. Yi and P. Fua, V. Lepetit
 ICCV 2015 paper – supplementary material
Detection and Fine 3D Pose Estimation of Texture-less Objects in RGB-D Images
T. Hodaň, X. Zabulis, M. Lourakis, Š. Obdržálek, J. Matas
 IROS 2015 paper
Learning Descriptors for Object Recognition and 3D Pose Estimation
P. Wohlhart, V. Lepetit
 CVPR 2015 paper – project page – video
Hashmod : A Hashing Method for Scalable 3D Object Detection
W. Kehl, F. Tombari, N. Navab, S. Ilic, V. Lepetit
 BMVC 2015 paper
Pose Estimation of Kinematic Chain Instances via Object Coordinate Regression
F. Michel, A. Krull, E. Brachmann, M. Y. Yang, S. Gumhold, C. Rother
 BMVC 2015 paper – project page – supplementary material – extended abstract
Latent-Class Hough Forests for 3D Object Detection and Pose Estimation
A. Tejani, D. Tang, R. Kouskouridas, T-K. Kim
 ECCV 2014 paper – project page – spotlight – video 1 – video 2
Learning 6D Object Pose Estimation using 3D Object Coordinates
E. Brachmann, A. Krull, F. Michel, S. Gumhold, J. Shotton, C. Rother
 ECCV 2014 paper – project page – supplementary material – spotlight – video 1
Robust Instance Recognition in Presence of Occlusion and Clutter
U. Bonde, V. Badrinarayanan, R. Cipolla
 ECCV 2014 paper – video
Model Based Training, Detection and Pose Estimation of Texture-Less 3D Objects in Heavily Cluttered Scenes
S. Hinterstoisser, V. Lepetit, S. Ilic, S. Holzer, G. R. Bradski, K. Konolige, N. Navab
 ACCV 2012 paper – video
Model Globally, Match Locally: Efficient and Robust 3D Object Recognition
B. Drost, M. Ulrich, N. Navab, S. Ilic
 CVPR 2010 paper

Title Notes
Imperial College London Multi-instance & Bin-picking Object Dataset
Technical University Dresden Articulated Object Dataset – Occluded Object Dataset
EPFL & Technical University Graz 3D Rigid Tracking from RGB Images Dataset
University of Birmingham Highly Occluded Object Dataset
University of Cambridge Desk3D Dataset
Rutgers University Rutgers APC RGB-D Dataset
Technical University of Munich LINEMOD Dataset
Czech Technical University T-LESS Dataset


If you believe that something is missing, please contact: