Skip to Main content Skip to Navigation

Modèle d'interaction et performances du traitement du signal multimodal

Abstract : The joint processing of multimodal measurements is supposed to lead to better performances than those obtained using a single modality or several modalities independently. However, in literature, there are examples that show that is not always true. In this thesis, we analyze, in terms of mutual information and estimation error, the different situations of multimodal analysis in order to determine the conditions to achieve the optimal performances.In the first part, we consider the simple case of two or three modalities, each associated with noisy measurement of a signal. These modalities are linked through the correlations between the useful parts of the signal and the correlations between the noises. We show that the performances are improved if the links between the modalities are exploited. In the second part, we study the impact on performance of wrong links between modalities. We show that these false assumptions decline the performance, which can become lower than the performance achieved using a single modality.In the general case, we model the multiple modalities as a noisy Gaussian channel. We then extend literature results by considering the impact of the errors on signal and noise probability densities on the information transmitted by the channel. We then analyze this relationship in the case of a simple model of two modalities. Our results show in particular the unexpected fact that a double mismatch of the noise and the signal can sometimes compensate for each other, and thus lead to very good performances.
Document type :
Complete list of metadatas

Cited literature [57 references]  Display  Hide  Download
Contributor : Abes Star :  Contact
Submitted on : Tuesday, September 25, 2018 - 10:49:07 AM
Last modification on : Wednesday, October 14, 2020 - 4:17:15 AM
Long-term archiving on: : Wednesday, December 26, 2018 - 1:15:59 PM


Version validated by the jury (STAR)


  • HAL Id : tel-01880756, version 1



Saloua Chlaily. Modèle d'interaction et performances du traitement du signal multimodal. Traitement du signal et de l'image [eess.SP]. Université Grenoble Alpes, 2018. Français. ⟨NNT : 2018GREAT026⟩. ⟨tel-01880756⟩



Record views


Files downloads