Traitement automatique des langues pour l'indexation d'images

Pierre Tirilly 1
1 TEXMEX - Multimedia content-based indexing
IRISA - Institut de Recherche en Informatique et Systèmes Aléatoires, Inria Rennes – Bretagne Atlantique
Abstract : Although it is globally in line with traditional information retrieval (IR), image indexing makes poor use of the existing work about textual IR and natural language processing (NLP). We identify two levels where such work could become integrated to image indexing systems. The first level is the description of the visual content of images. To integrate NLP at this level, we adopt a visual word-based representation of images, as proposed by Sivic and Zisserman. This representation raises two issues that are classical in textual IR: choosing relevant index terms and taking into account the relations between index terms. We address the first issue by studying stop-lists and weighting schemes in the context of image indexing. Our experiments show that there is no optimal weighting scheme in the general case, and that it should be chosen in keeping with the query. Then, we address the second issue by adapting language models to images, to go beyond the term independence hypothesis. Our experiments show that, in the context of image classification, taking account of spatial relations between visual words can improve the systems' performances. The second level where we integrate NLP to image indexing is semantic image indexing: we can use NLP techniques on texts coming with images to extract a textual description of these images. We first show that standard image descriptors are not suited to image annotation, then we propose an image annotation scheme that avoid this problem by using high-level textual and visual concepts: we extract named entities from texts and associate them with visual concepts that we detect in the images. We validate our approach on a real-world and large-scale news corpus.
Document type :
Complete list of metadatas
Contributor : Pierre Tirilly <>
Submitted on : Thursday, September 9, 2010 - 3:36:48 PM
Last modification on : Friday, November 16, 2018 - 1:21:56 AM
Long-term archiving on : Tuesday, October 23, 2012 - 3:50:12 PM


  • HAL Id : tel-00516422, version 1


Pierre Tirilly. Traitement automatique des langues pour l'indexation d'images. Interface homme-machine [cs.HC]. Université Rennes 1, 2010. Français. ⟨tel-00516422⟩



Record views


Files downloads