Sequential Learning with Similarities

Tomáš Kocák 1
1 SEQUEL - Sequential Learning
Inria Lille - Nord Europe, CRIStAL - Centre de Recherche en Informatique, Signal et Automatique de Lille (CRIStAL) - UMR 9189
Abstract : This thesis studies several extensions of multi-armed bandit problem, where a learner sequentially selects an action and obtains the reward of the action. Traditionally, the only information the learner acquires is about the obtained reward while information about other actions is hidden from the learner. This limited feedback can be restrictive in some applications like recommender systems, internet advertising, packet routing, etc. Usually, these problems come with structure, similarities between users or actions, additional observations, or any additional assumptions. Therefore, it is natural to incorporate these assumptions to the algorithms to improve their performance. This thesis focuses on multi-armed bandit problem with some underlying structure usually represented by a graph with actions as vertices. First, we study a problem where the graph captures similarities between actions; connected actions tend to grand similar rewards. Second, we study a problem where the learner observes rewards of all the neighbors of the selected action. We study these problems under several additional assumptions on rewards (stochastic, adversarial), side observations (adversarial, stochastic, noisy), actions (one node at the time, several nodes forming a combinatorial structure in the graph). The main contribution of this thesis is to design algorithms for previously mentioned problems together with theoretical and empirical guarantees. We also introduce several novel quantities, to capture the difficulty of some problems, like effective dimension and effective independence number.
Document type :
Theses
Complete list of metadatas

Cited literature [90 references]  Display  Hide  Download

https://tel.archives-ouvertes.fr/tel-01742570
Contributor : Tomáš Kocák <>
Submitted on : Sunday, March 25, 2018 - 4:47:25 PM
Last modification on : Friday, May 17, 2019 - 11:40:03 AM
Long-term archiving on : Thursday, September 13, 2018 - 7:53:53 AM

File

Thesis.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : tel-01742570, version 1

Citation

Tomáš Kocák. Sequential Learning with Similarities. Machine Learning [cs.LG]. Inria Lille Nord Europe - Laboratoire CRIStAL - Université de Lille, 2016. English. ⟨tel-01742570⟩

Share

Metrics

Record views

201

Files downloads

203