Stratégie de fusion pour des signaux écrits et sonores : Application à la reconnaissance d'expressions mathématiques

Sofiane Medjkoune 1
1 irccyn-ivc
IRCCyN - Institut de Recherche en Communications et en Cybernétique de Nantes
Abstract : Significant efforts are being done to make as natural as possible the way that human are interacting with their machines. Regarding this quest, a lot of research is being inspired by the most sophisticated machine ever known : human being and more precisely his use of the multi-modality aspect of the information to interact with his peers. The work reported here concerns the study, the conception and the validation of bidimensional structure recognition systems. The application considered here is the mathematical expression language which is one of the most interesting 2D languages. The system we proposed is original since it uses simultaneously two modalities to achieve its task. Indeed, both speech and handwriting streams are used by our system to perform the recognition in a bimodal fashion. This procedure allows dealing with the ambiguities arising when mono-modal processing is used. This system exploits the existing complementarity between the modalities in concern and exhibits an improvement of the performances with respect to the case of a mono-modal processing using only handwriting modality. To set-up, train and validate our system we built HAMEX, a bimodal database of mathematical expressions. This latter, is formed by 4350 mathematical expressions, each available in handwritten and audio forms and is fully annotated.
Complete list of metadatas

Cited literature [29 references]  Display  Hide  Download
Contributor : Harold Mouchère <>
Submitted on : Wednesday, March 25, 2015 - 4:51:43 PM
Last modification on : Wednesday, December 19, 2018 - 3:02:08 PM
Long-term archiving on : Thursday, July 2, 2015 - 7:34:46 AM


  • HAL Id : tel-01135694, version 1



Sofiane Medjkoune. Stratégie de fusion pour des signaux écrits et sonores : Application à la reconnaissance d'expressions mathématiques. Traitement du signal et de l'image [eess.SP]. Université de Nantes, 2013. Français. ⟨NNT : ED 503-206⟩. ⟨tel-01135694⟩



Record views


Files downloads