Skip to Main content Skip to Navigation

Représentations visuelles de concepts textuels pour la recherche et l'annotation interactives d'images

Abstract : As regard image retrieval today, we often manipulate large volumes of images, which may vary or even update continuously. In an image database, we end up with both old and new images, the first possibly already indexed and annotated and the latter waiting for indexing or annotation. Since the database is not annotated consistently, it is difficult to use text queries. We present in this work different techniques to interact, navigate and search in this type of image databases. First, a model for short term interaction is used to improve the accuracy of the system. Second, based on a model of long terminteraction, we propose to combine semantic concepts and visual features to search for images by text, visual content or a mix between text and visual content. This model of image retrieval can iteratively refine the annotation of images.We identify four contributions in this work. The first contribution is a system for multimodal retrieval of images which includes different kinds of data, like visual content and text. This system can be queried by images, by keywords or by hybrid text/visual queries. The second contribution is a novel technique of relevance feedback combining 2 classic techniques: query point movement and query expansion. This technique profits for non-pertinent feedback and combines the advantages of both classic techniques and improve performance for interactive image retrieval. The third contribution is a model based on visual representations of keywords (KVR: Keyword Visual Representation) that create links between textand visual content, based on long term interaction. With the strategy of incremental learning, this model provides an association between semantic concepts and visual features that help improve the accuracy of image annotation and image retrieval. Moreover, the visual representation of textual concept gives users the ability to query the system by text queries or mixed queries text / images, even if the image database is only partially annotated. The fourth contribution, under the assumption that knowledge is not available early in most image retrieval systems, is a mechanism for incremental construction of knowledge from scratch. We do not separate phases of retrieval and annotation, and the user can makequeries from the start of the system, while allowing the system to learn incrementally when it is used. The contributions above are completed by an interface for viewing and querying mixing textual and visual content. Although at present only two types of information are used, the text and visual content, the genericity of the proposed model allows its extension to other types of external information, such as location (GPS) and time.
Document type :
Complete list of metadatas

Cited literature [114 references]  Display  Hide  Download
Contributor : Abes Star :  Contact
Submitted on : Monday, September 10, 2012 - 11:22:17 PM
Last modification on : Wednesday, October 14, 2020 - 3:55:20 AM
Long-term archiving on: : Tuesday, December 11, 2012 - 3:44:20 AM


Version validated by the jury (STAR)


  • HAL Id : tel-00730707, version 1



Nhu Van Nguyen. Représentations visuelles de concepts textuels pour la recherche et l'annotation interactives d'images. Ordinateur et société [cs.CY]. Université de La Rochelle, 2011. Français. ⟨NNT : 2011LAROS338⟩. ⟨tel-00730707⟩



Record views


Files downloads