Skip to Main content Skip to Navigation

Upper body tracking and Gesture recognition for Human-Machine Interaction

I. Renna 1
1 Interaction
ISIR - Institut des Systèmes Intelligents et de Robotique
Abstract : Robots are artificial agents that can act in humans' world thanks to perception, action and reasoning capacities. In particular, robots companion are designed to share with humans the same physical and communication spaces in performing daily life collaborative tasks and aids. In such a context, interactions between humans and robots are expected to be as natural and as intuitive as possible. One of the most natural ways is based on gestures and reactive body motions. To make this friendly interaction possible, a robot companion has to be endowed with one or more capabilities allowing him to perceive, to recognize and to react to human gestures. This PhD thesis has been focused on the design and the development of a gesture recognition system that can be exploited in a human-robot interaction context. This system includes (1) a limbs-tracking algorithm that determines human body position during movements and (2) a higher-level module that recognizes gestures performed by human users. New contributions were made in both topics. First, a new approach is proposed for visual tracking of upper-body limbs. Analyzing human body motion is challenging, due to the important number of degrees of freedom of the articulated object modeling the upper body. To circumvent the computational complexity, each limb is tracked with an Annealed Particle Filter and the different filters interact through Belief Propagation. 3D human body is described as a graphical model in which the relationships between the body parts are represented by conditional probability distributions. Pose estimation problem is thus formulated as a probabilistic inference over a graphical model, where the random variables correspond to the individual limb parameters (position and orientation) and Belief Propagation messages ensure coherence between limbs. Secondly, we propose a framework allowing emblematic gestures detection and recognition. The most challenging issue in gesture recognition is to find good features with a discriminant power (to distinguish between different gestures) and a good robustness to intrinsic gestures variability (the context in which gestures are expressed, the morphology of the person, the point of view, etc.). In this work, we propose a new arm's kinematics normalization scheme reflecting both the muscular activity and arm's appearance when a gesture is performed. The obtained signals are first segmented and then analyzed by two machine learning techniques: Hidden Markov Models and Support Vector Machines. The two methods are com- pared in a 5 classes emblematic gestures recognition task. Both systems show good performances with a minimalistic training database regardless to performer's anthropometry, gender, age or pose with regard to the sensing system. The work presented here has been done within the framework of a PhD thesis in joint supervision between the "Pierre et Marie Curie" University (ISIR laboratory, Paris) and the University of Genova (IIT-Tera department) and was labeled by the French-Italian University.
Document type :
Complete list of metadatas

Cited literature [243 references]  Display  Hide  Download
Contributor : Ilaria Renna <>
Submitted on : Friday, July 13, 2012 - 2:00:37 PM
Last modification on : Monday, December 14, 2020 - 9:52:50 AM


  • HAL Id : tel-00717443, version 1


I. Renna. Upper body tracking and Gesture recognition for Human-Machine Interaction. Robotique [cs.RO]. Université Pierre et Marie Curie - Paris VI, 2012. Français. ⟨tel-00717443⟩



Record views


Files downloads