Skip to Main content Skip to Navigation

Sequential Learning with Similarities

Tomáš Kocák 1
1 SEQUEL - Sequential Learning
Inria Lille - Nord Europe, CRIStAL - Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189
Abstract : This thesis studies several extensions of multi-armed bandit problem, where a learner sequentially selects an action and obtains the reward of the action. Traditionally, the only information the learner acquires is about the obtained reward while information about other actions is hidden from the learner. This limited feedback can be restrictive in some applications like recommender systems, internet advertising, packet routing, etc. Usually, these problems come with structure, similarities between users or actions, additional observations, or any additional assumptions. Therefore, it is natural to incorporate these assumptions to the algorithms to improve their performance. This thesis focuses on multi-armed bandit problem with some underlying structure usually represented by a graph with actions as vertices. First, we study a problem where the graph captures similarities between actions; connected actions tend to grand similar rewards. Second, we study a problem where the learner observes rewards of all the neighbors of the selected action. We study these problems under several additional assumptions on rewards (stochastic, adversarial), side observations (adversarial, stochastic, noisy), actions (one node at the time, several nodes forming a combinatorial structure in the graph). The main contribution of this thesis is to design algorithms for previously mentioned problems together with theoretical and empirical guarantees. We also introduce several novel quantities, to capture the difficulty of some problems, like effective dimension and effective independence number.
Document type :
Complete list of metadata

Cited literature [90 references]  Display  Hide  Download
Contributor : Tomáš Kocák Connect in order to contact the contributor
Submitted on : Sunday, March 25, 2018 - 4:47:25 PM
Last modification on : Thursday, January 20, 2022 - 4:16:37 PM
Long-term archiving on: : Thursday, September 13, 2018 - 7:53:53 AM


Files produced by the author(s)


  • HAL Id : tel-01742570, version 1


Tomáš Kocák. Sequential Learning with Similarities. Machine Learning [cs.LG]. Inria Lille Nord Europe - Laboratoire CRIStAL - Université de Lille, 2016. English. ⟨tel-01742570⟩



Les métriques sont temporairement indisponibles