Skip to Main content Skip to Navigation

Reconnaissance de la Langue Française Parlée Complété (LPC) : décodage phonétique des gestes main-lèvres.

Abstract : Cued Speech (CS) is a visual communication system that uses handshapes placed in different positions near the face, in combination with the natural speech lip-reading, to enhance speech perception from visual input for deaf people. In this system, the speaker moves his hand in close relation with speech. Handshapes are designed to distinguish among consonants whereas hand positions are used to distinguish among vowels. Due to the CS system, both manual and lip flows produced by the CS speaker carry a part of the phonetic information. This work presents at first a method for the automatic coding of the manual flow in term of CS hand positions and CS handshapes. Then the lip-shape classification of the vowels and the consonants is discussed. The labial flow is composed of the temporal variations of lip parameters extracted from the inner and the outer contours of the lips. This work will show how the distribution of lip parameters inside each group of CS hand positions allows vowel discrimination. A classification method based on Gaussian modeling is presented and results demonstrate a good performance of this classification (89% as test score). The vocalic context is taken into account in the case of the consonants, with the use of HMM for the modeling of the lip transition from the consonant towards the vowel (80 % as test scores in term of CV visemes). Finally, the modeling of the lip information and the coding of the manual flow are included in a “Master-Slave” fusion model for recognition of the vowels and the consonants in the CS context. The fusion model integrates the temporal constraints of the CS production and perception. This work is thus also a first contribution to the modeling of the CS system from the perceptive point of view.
Document type :
Domain :
Complete list of metadata

Cited literature [224 references]  Display  Hide  Download
Contributor : Noureddine Aboutabit <>
Submitted on : Thursday, April 3, 2008 - 5:51:08 PM
Last modification on : Tuesday, August 24, 2021 - 11:42:06 AM
Long-term archiving on: : Friday, May 21, 2010 - 1:19:17 AM


  • HAL Id : tel-00270162, version 1


Noureddine Aboutabit. Reconnaissance de la Langue Française Parlée Complété (LPC) : décodage phonétique des gestes main-lèvres.. domain_stic. Institut National Polytechnique de Grenoble - INPG, 2007. Français. ⟨tel-00270162⟩



Record views


Files downloads