Skip to Main content Skip to Navigation

Tout est dans le regard : reconnaissance visuelle du comportement humain en vue subjective

Abstract : In this thesis, we focus on understanding human behavior from gaze information. In contrast to static and external camera viewpoint, we adopt a first-person point of view that allows carrying studies centered on humans and their interaction with the environment. To fulfill this goal, we developed a head-mounted eye-tracker and analysis tools for attention recognition during social interactions and for gaze-based first-person activity recognition. In the first part of the thesis, we present a head-mounted binocular eye-tracker from which we infer the subject's gaze. Contrary to infrared systems, our approach works under visible light. Instead of extracting geometric features (e.g. pupil), we propose to use an eye appearance model in order to capture all available eye features. To learn the mapping between eye appearance and point of regard, two regression models are compared: Support Vector Regression and Relevance Vector Regression. Then, we propose a novel approach for attention recognition from first-person vision. The first-person gaze is obtained using our eye-tracker, while the third-person gaze is computed from head pose estimation based on localized multiple kernel regression. Knowing the first- and third-person gaze direction, scores are computed which permit to assign dyadic attention patterns such as mutual gaze, and at the same time, higher-order patterns due to the triadic nature of the experiment. Our final analysis tool involves activity recognition from first-person gaze and egomotion. These motions are quantized according to their direction and their amplitude, and are encoded into a sequence of symbols. Statistical features are then extracted via multi-scale and temporal representation. For joint classification and segmentation of activities, we describe a contextual learning approach built upon confidence values from long-range neighborhood. Additionally, an in-depth study allows highlighting which features are relevant to each activity.
Complete list of metadatas

Cited literature [189 references]  Display  Hide  Download
Contributor : Francis Martinez <>
Submitted on : Thursday, June 5, 2014 - 12:07:06 AM
Last modification on : Friday, March 22, 2019 - 1:32:16 AM
Document(s) archivé(s) le : Friday, September 5, 2014 - 11:00:42 AM


  • HAL Id : tel-01001816, version 1


Francis Martinez. Tout est dans le regard : reconnaissance visuelle du comportement humain en vue subjective. Vision par ordinateur et reconnaissance de formes [cs.CV]. Université Pierre et Marie Curie - Paris VI, 2013. Français. ⟨tel-01001816⟩



Record views


Files downloads