The Emergence of Multimodal Concepts : From Perceptual Motion Primitives to Grounded Acoustic Words

Olivier Mangin 1
1 Flowers - Flowing Epigenetic Robots and Systems
Inria Bordeaux - Sud-Ouest, U2IS - Unité d'Informatique et d'Ingénierie des Systèmes
Abstract : This thesis focuses on learning recurring patterns in multimodal perception. For that purpose it develops cognitive systems that model the mechanisms providing such capabilities to infants; a methodology that fits into thefield of developmental robotics.More precisely, this thesis revolves around two main topics that are, on the one hand the ability of infants or robots to imitate and understand human behaviors, and on the other the acquisition of language. At the crossing of these topics, we study the question of the how a developmental cognitive agent can discover a dictionary of primitive patterns from its multimodal perceptual flow. We specify this problem and formulate its links with Quine's indetermination of translation and blind source separation, as studied in acoustics.We sequentially study four sub-problems and provide an experimental formulation of each of them. We then describe and test computational models of agents solving these problems. They are particularly based on bag-of-words techniques, matrix factorization algorithms, and inverse reinforcement learning approaches. We first go in depth into the three separate problems of learning primitive sounds, such as phonemes or words, learning primitive dance motions, and learning primitive objective that compose complex tasks. Finally we study the problem of learning multimodal primitive patterns, which corresponds to solve simultaneously several of the aforementioned problems. We also details how the last problems models acoustic words grounding.
Document type :
Theses
Complete list of metadatas

Cited literature [198 references]  Display  Hide  Download

https://tel.archives-ouvertes.fr/tel-01148936
Contributor : Abes Star <>
Submitted on : Tuesday, May 5, 2015 - 4:57:06 PM
Last modification on : Saturday, October 5, 2019 - 3:40:06 AM
Long-term archiving on: Wednesday, April 19, 2017 - 5:27:07 PM

File

MANGIN_OLIVIER_2014.pdf
Version validated by the jury (STAR)

Identifiers

  • HAL Id : tel-01148936, version 1

Collections

Citation

Olivier Mangin. The Emergence of Multimodal Concepts : From Perceptual Motion Primitives to Grounded Acoustic Words. Other [cs.OH]. Université de Bordeaux, 2014. English. ⟨NNT : 2014BORD0002⟩. ⟨tel-01148936⟩

Share

Metrics

Record views

2886

Files downloads

825