Skip to Main content Skip to Navigation

Evaluating exploratory search engines : designing a set of user-centered methods based on a modeling of the exploratory search process

Emilie Palagi 1, 2 
1 WIMMICS - Web-Instrumented Man-Machine Interactions, Communities and Semantics
CRISAM - Inria Sophia Antipolis - Méditerranée , Laboratoire I3S - SPARKS - Scalable and Pervasive softwARe and Knowledge Systems
Abstract : Exploratory search systems are search engines that help users to explore a topic of interest. A shortcoming of current evaluation methods is that they cannot be used to determine if an exploratory search system can effectively help the user in performing exploratory search tasks. Indeed, the assessment cannot be the same between classic search systems (such as Google, Bing, Yahoo!...) and exploratory search systems. The complexity and the difficulty to have a consensus definition of the exploratory search concept and process are reflected in the difficulties to evaluate such systems. Indeed, they combine several specifics features and behaviors forming an alchemy difficult to evaluate. The main objective of this thesis is to propose for the designers of these systems (i.e. computer scientists) user-centered evaluation methods of exploratory search systems. These methods are based on a model of exploratory search process in order to help the evaluators to verify if a given system supports effectively the exploratory search process. Thus, after elaborating a model of exploratory search process, we propose two model-based methods, with and without users, which can be used all along the design process. The first method, without users, can be used from the first sketch of the system, consists of a set of heuristics of exploratory search and a procedure for using them. We also propose two tools facilitating their use: an online form format and an Google Chrome plugin, CheXplore. The second method involves real end-users of exploratory search systems who test a functional prototype or version of an exploratory search system. In this thesis, we mainly focus on two model-based elements of a customizable user testing procedure: a protocol for the elaboration of exploratory search tasks and a video analysis grid for the evaluation of recorded exploratory search sessions.
Complete list of metadata

Cited literature [81 references]  Display  Hide  Download
Contributor : ABES STAR :  Contact
Submitted on : Wednesday, February 20, 2019 - 3:26:08 PM
Last modification on : Thursday, August 4, 2022 - 4:55:00 PM


Version validated by the jury (STAR)


  • HAL Id : tel-01976017, version 3



Emilie Palagi. Evaluating exploratory search engines : designing a set of user-centered methods based on a modeling of the exploratory search process. Human-Computer Interaction [cs.HC]. Université Côte d'Azur, 2018. English. ⟨NNT : 2018AZUR4116⟩. ⟨tel-01976017v3⟩



Record views


Files downloads