Evaluating exploratory search engines : designing a set of user-centered methods based on a modeling of the exploratory search process

Emilie Palagi 1, 2
1 WIMMICS - Web-Instrumented Man-Machine Interactions, Communities and Semantics
CRISAM - Inria Sophia Antipolis - Méditerranée , Laboratoire I3S - SPARKS - Scalable and Pervasive softwARe and Knowledge Systems
Abstract : Exploratory search systems are search engines that help users to explore a topic of interest. A shortcoming of current evaluation methods is that they cannot be used to determine if an exploratory search system can effectively help the user in performing exploratory search tasks. Indeed, the assessment cannot be the same between classic search systems (such as Google, Bing, Yahoo!...) and exploratory search systems. The complexity and the difficulty to have a consensus definition of the exploratory search concept and process are reflected in the difficulties to evaluate such systems. Indeed, they combine several specifics features and behaviors forming an alchemy difficult to evaluate. The main objective of this thesis is to propose for the designers of these systems (i.e. computer scientists) user-centered evaluation methods of exploratory search systems. These methods are based on a model of exploratory search process in order to help the evaluators to verify if a given system supports effectively the exploratory search process. Thus, after elaborating a model of exploratory search process, we propose two model-based methods, with and without users, which can be used all along the design process. The first method, without users, can be used from the first sketch of the system, consists of a set of heuristics of exploratory search and a procedure for using them. We also propose two tools facilitating their use: an online form format and an Google Chrome plugin, CheXplore. The second method involves real end-users of exploratory search systems who test a functional prototype or version of an exploratory search system. In this thesis, we mainly focus on two model-based elements of a customizable user testing procedure: a protocol for the elaboration of exploratory search tasks and a video analysis grid for the evaluation of recorded exploratory search sessions.
Complete list of metadatas

https://tel.archives-ouvertes.fr/tel-01976017
Contributor : Abes Star <>
Submitted on : Wednesday, February 20, 2019 - 3:26:08 PM
Last modification on : Wednesday, April 17, 2019 - 9:57:10 AM

File

2018AZUR4116.pdf
Version validated by the jury (STAR)

Identifiers

  • HAL Id : tel-01976017, version 3

Collections

Citation

Emilie Palagi. Evaluating exploratory search engines : designing a set of user-centered methods based on a modeling of the exploratory search process. Human-Computer Interaction [cs.HC]. Université Côte d'Azur, 2018. English. ⟨NNT : 2018AZUR4116⟩. ⟨tel-01976017v3⟩

Share

Metrics

Record views

178

Files downloads

85