Service interruption on Monday 11 July from 12:30 to 13:00: all the sites of the CCSD (HAL, Epiciences, SciencesConf, AureHAL) will be inaccessible (network hardware connection).
Skip to Main content Skip to Navigation

Semantic Description of Humans in Images

Gaurav Sharma 1, 2 
1 LEAR - Learning and recognition in vision
Inria Grenoble - Rhône-Alpes, LJK - Laboratoire Jean Kuntzmann, Grenoble INP - Institut polytechnique de Grenoble - Grenoble Institute of Technology
2 Equipe Image - Laboratoire GREYC - UMR6072
GREYC - Groupe de Recherche en Informatique, Image et Instrumentation de Caen
Abstract : In the present thesis we are interested in semantic description of humans in images. We propose to describe humans with the help of (i) semantic attributes e.g. male or female, wearing a tee-shirt, (ii) actions e.g. riding a horse, running and (iii) facial expressions e.g. smiling, angry. First, we propose a new image representation to better exploit the class specific spatial information. The standard representation \ie spatial pyramids, has two shortcomings. It assumes that the distribution of spatial information (i) is uniform and (ii) is same for all tasks. We address these shortcomings by learning the discriminative spatial information for a specific task. Further, we propose a model that adapts the spatial information for each image for a given task. This lends more flexibility to the model and allows for misalignments of discriminative regions e.g. the legs may be at different positions, in different images for running class. Finally, we propose a new descriptor for facial expression analysis. We work in the space of intensity differences of local pixel neighborhoods and propose to learn the quantization of the space and use higher order statistics of the difference vector to obtain more expressive descriptors. We introduce a challenging dataset of human attributes containing 9344 human images, sourced from the internet, with annotations for 27 semantic attributes based on sex, pose, age and appearance/clothing. We validate the proposed methods on our dataset of human attributes as well as on publicly available datasets of human actions, fine grained classification involving human actions and facial expressions. We also report results on related computer vision datasets, for scene recognition, object image classification and texture categorization, to highlight the generality of our contributions.
Complete list of metadata

Cited literature [116 references]  Display  Hide  Download
Contributor : THOTH Team Connect in order to contact the contributor
Submitted on : Thursday, December 20, 2012 - 12:07:57 PM
Last modification on : Saturday, June 25, 2022 - 9:47:00 AM
Long-term archiving on: : Thursday, March 21, 2013 - 3:48:51 AM


  • HAL Id : tel-00767699, version 1


Gaurav Sharma. Semantic Description of Humans in Images. Computer Vision and Pattern Recognition [cs.CV]. Université de Caen, 2012. English. ⟨tel-00767699⟩



Record views


Files downloads