HAL will be down for maintenance from Friday, June 10 at 4pm through Monday, June 13 at 9am. More information
Skip to Main content Skip to Navigation
Theses

Gaze and face-to-face communication between a human speaker and an animated conversational agent - Mutual attention and multimodal deixis

Abstract : In the context of synthetic generation and decoding of linguistic information, not only the audible component but also the visual component of speech conveys valuable information. We address gaze as an important modality to enhance speech and to convey additional information. Gaze is an important deictic gesture as well as it plays various roles in the organization of dialogue and social interaction. In a first experiment, we investigated how the gaze of a talking head can be used as a deictic gesture in an on-screen search and retrieval task. We found that such gestures are appropriate to reduce processing time as well as cognitive load. Multimodal gestures incorporating speech in a coherent way showed to be more efficient than only visual gestures. In a second experiment, we investigated the relations between the gaze of a target subject and different elements of conversational interaction. We defined different stages in the dialogic exchange of information and found that these are related to the variations in the measured gaze behavior. Based on the observed characteristics we propose a model to control the gaze of an embodied conversational agent in close dyadic interaction.
Complete list of metadata

Cited literature [135 references]  Display  Hide  Download

https://tel.archives-ouvertes.fr/tel-00369479
Contributor : Gérard Bailly Connect in order to contact the contributor
Submitted on : Friday, March 20, 2009 - 9:16:47 AM
Last modification on : Friday, March 25, 2022 - 11:09:52 AM
Long-term archiving on: : Thursday, June 10, 2010 - 5:21:26 PM

Identifiers

  • HAL Id : tel-00369479, version 1

Citation

Stephan Raidt. Gaze and face-to-face communication between a human speaker and an animated conversational agent - Mutual attention and multimodal deixis. Signal and Image processing. Institut National Polytechnique de Grenoble - INPG, 2008. English. ⟨tel-00369479⟩

Share

Metrics

Record views

234

Files downloads

353