Skip to Main content Skip to Navigation

Language Modelling for Handwriting Recognition

Abstract : This thesis is about the design of a complete processing chain dedicated to unconstrained handwriting recognition. Three main difficulties are adressed: pre-processing, optical modeling and language modeling. The pre-processing stage is related to extracting properly the text lines to be recognized from the document image. An iterative text line segmentation method using oriented steerable filters was developed for this purpose. The difficulty in the optical modeling stage lies in style diversity of the handwriting scripts. Statistical optical models are traditionally used to tackle this problem such as Hidden Markov models (HMM-GMM) and more recently recurrent neural networks (BLSTM-CTC). Using BLSTM we achieve state of the art performance on the RIMES (for French) and IAM (for English) datasets. The language modeling stage implies the integration of a lexicon and a statistical language model to the recognition processing chain in order to constrain the recognition hypotheses to the most probable sequence of words (sentence) from the language point of view. The difficulty at this stage is related to the finding the optimal vocabulary with minimum Out-Of-Vocabulary words rate (OOV). Enhanced language modeling approaches has been introduced by using sub-lexical units made of syllables or multigrams. The sub-lexical units cover an important portion of the OOV words. Then the language coverage depends on the domain of the language model training corpus, thus the need to train the language model with in domain data. The recognition system performance with the sub-lexical units outperformes the traditional recognition systems that use words or characters language models, in case of high OOV rates. Otherwise equivalent performances are obtained with a compact sub-lexical language model. Thanks to the compact lexicon size of the sub-lexical units, a unified multilingual recognition system has been designed. The unified system performance have been evaluated on the RIMES and IAM datasets. The unified multilingual system shows enhanced recognition performance over the specialized systems, especially when a unified optical model is used.
Document type :
Complete list of metadatas

Cited literature [199 references]  Display  Hide  Download
Contributor : Abes Star :  Contact
Submitted on : Monday, April 30, 2018 - 1:01:45 AM
Last modification on : Thursday, November 28, 2019 - 4:03:42 AM
Long-term archiving on: : Thursday, September 20, 2018 - 3:07:50 AM


Version validated by the jury (STAR)


  • HAL Id : tel-01781268, version 1


Wassim Swaileh. Language Modelling for Handwriting Recognition. Modeling and Simulation. Normandie Université, 2017. English. ⟨NNT : 2017NORMR024⟩. ⟨tel-01781268⟩



Record views


Files downloads