Skip to Main content Skip to Navigation

Representing 3D models for alignment and recognition

Abstract : Thanks to the success of 3D reconstruction algorithms and the development of online tools for computer-aided design (CAD) the number of publicly available 3D models has grown significantly in recent years, and will continue to do so. This thesis investigates representations of 3D models for 3D shape matching, instance-level 2D-3D alignment, and category-level 2D-3D recognition. The geometry of a 3D shape can be represented almost completely by the eigen-functions and eigen-values of the Laplace-Beltrami operator on the shape. We use this mathematically elegant representation to characterize points on the shape, with a new notion of scale. This 3D point signature can be interpreted in the framework of quantum mechanics and we call it the Wave Kernel Signature (WKS). We show that it has advantages with respect to the previous state-of-the-art shape descriptors, and can be used for 3D shape matching, segmentation and recognition. A key element for understanding images is the ability to align an object depicted in an image to its given 3D model. We tackle this instance level 2D-3D alignment problem for arbitrary 2D depictions including drawings, paintings, and historical photographs. This is a tremendously diffcult task as the appearance and scene structure in the 2D depictions can be very different from the appearance and geometry of the 3D model, e.g., due to the specific rendering style, drawing error, age, lighting or change of seasons. We represent the 3D model of an entire architectural site by a set of visual parts learned from rendered views of the site. We then develop a procedure to match those scene parts that we call 3D discriminative visual elements to the 2D depiction of the architectural site. We validate our method on a newly collected dataset of non-photographic and historical depictions of three architectural sites. We extend this approach to describe not only a single architectural site but an entire object category, represented by a large collection of 3D CAD models. We develop a category-level 2D-3D alignment method that not only detects objects in cluttered images but also identifies their approximate style and viewpoint. We evaluate our approach both qualitatively and quantitatively on a subset of the challenging Pascal VOC 2012 images of the \chair" category using a reference library of 1394 CAD models downloaded from the Internet.
Complete list of metadata

Cited literature [192 references]  Display  Hide  Download
Contributor : ABES STAR :  Contact
Submitted on : Wednesday, April 11, 2018 - 5:19:08 PM
Last modification on : Friday, June 24, 2022 - 3:11:22 AM


Version validated by the jury (STAR)


  • HAL Id : tel-01160300, version 2



Mathieu Aubry. Representing 3D models for alignment and recognition. Computer Vision and Pattern Recognition [cs.CV]. Ecole normale supérieure - ENS PARIS, 2015. English. ⟨NNT : 2015ENSU0006⟩. ⟨tel-01160300v2⟩



Record views


Files downloads