Y. Abbasi-yadkori, D. Pal, and C. Szepesvari, Improved algorithms for linear stochastic bandits, Advances in Neural Information Processing Systems 24 th (NIPS), pp.2312-2320, 2011.

N. Abe and A. Nakamura, Learning to Optimally Schedule Internet Banner Advertisements, Proc. of the 16 th International Conference on Machine Learning (ICML), pp.12-21, 1999.

J. Abernethy, F. Bach, T. Evgeniou, and J. Vert, Low-rank matrix factorization with attributes, Inria tech Report, 2006.

M. Adams and A. B. Nobel, Uniform approximation of Vapnik???Chervonenkis classes, Bernoulli, vol.18, issue.4, pp.1310-1319, 2012.
DOI : 10.3150/11-BEJ379

G. S. All and O. C. , Kdd cup 2012 Kaggle criteo ad display challenge, 2012.

J. Audibert, R. Munos, and C. Szepesvári, Exploration???exploitation tradeoff using variance estimates in multi-armed bandits, Theoretical Computer Science, vol.410, issue.19, pp.4101876-1902, 2009.
DOI : 10.1016/j.tcs.2009.01.016
URL : https://hal.archives-ouvertes.fr/hal-00711069

P. Auer, N. Cesa-bianchi, and P. Fischer, Finite-time analysis of the multiarmed bandit problem, Machine Learning, vol.47, issue.2/3, pp.235-256, 2002.
DOI : 10.1023/A:1013689704352

F. Balcan, N. Bansal, A. Beygelzimer, D. Coppersmith, J. L. Sorkin et al., Robust Reductions from Ranking to Classification, Lecture Notes in Computer Science, vol.4539, pp.604-619, 2007.
DOI : 10.1007/978-3-540-72927-3_43

J. Bobadilla, F. Ortega, A. Hernando, and A. Gutiérrez, Recommender systems survey, Knowledge-Based Systems, vol.46, pp.109-132, 2013.
DOI : 10.1016/j.knosys.2013.03.012

L. Bottou, Stochastic gradient tricks, Neural Networks, Tricks of the Trade, Reloaded, Lecture Notes in Computer Science (LNCS 7700), pp.430-445, 2012.

L. Bottou, J. Peters, J. Quiñonero-candela, D. X. Charles, D. M. Chickering et al., Counterfactual reasoning and learning systems, 2012.

M. Boullé, MODL: A Bayes optimal discretization method for continuous attributes, Machine Learning, pp.131-165, 2006.
DOI : 10.1007/s10994-006-8364-x

C. J. Burges, From ranknet to lambdarank to lambdamart: An overview, 2010.

L. Busoniu, D. Ernst, D. Schutter, B. Babuska, and R. , Approximate reinforcement learning: An overview, 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL), pp.1-8, 2011.
DOI : 10.1109/ADPRL.2011.5967353

J. Cai, E. J. Candès, and Z. Shen, A Singular Value Thresholding Algorithm for Matrix Completion, SIAM Journal on Optimization, vol.20, issue.4, pp.1956-1982, 2010.
DOI : 10.1137/080738970

E. J. Candès and B. Recht, Exact Matrix Completion via Convex Optimization, Foundations of Computational Mathematics, vol.170, issue.1, pp.717-772, 2009.
DOI : 10.1007/s10208-009-9045-5

E. J. Candes and T. Tao, Decoding by Linear Programming, IEEE Transactions on Information Theory, vol.51, issue.12, pp.4203-4215, 2005.
DOI : 10.1109/TIT.2005.858979

E. J. Candès and T. Tao, The Power of Convex Relaxation: Near-Optimal Matrix Completion, IEEE Transactions on Information Theory, vol.56, issue.5, pp.2053-2080, 2010.
DOI : 10.1109/TIT.2010.2044061

D. Chakrabarti, R. Kumar, F. Radlinski, and E. Upfal, Mortal Multi-Armed Bandits, 20 th Advances in Neural Information Processing Systems (NIPS), pp.273-280, 2008.

O. Chapelle, L. Li, J. Shawe-taylor, R. S. Zemel, P. L. Bartlett et al., An empirical evaluation of thompson sampling, Advances in Neural Information Processing Systems 24: Proceedings of the 25th Annual Conference on Neural Information Processing Systems (NIPS-2011), pp.2249-2257, 2011.

W. Chu and Z. Ghahramani, Probabilistic models for incomplete multi-dimensional arrays, Proceedings of the Twelfth International Conference on Artificial Intelligence and Statistics (AISTATS-09) Journal of Machine Learning Research -Proceedings Track, pp.89-96, 2009.

C. Ding, T. Li, W. Peng, and H. Park, Orthogonal nonnegative matrix t-factorizations for clustering, Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining , KDD '06, pp.126-135, 2006.
DOI : 10.1145/1150402.1150420

J. Duchi, E. Hazan, and Y. Singer, Adaptive subgradient methods for online learning and stochastic optimization, 2010.

M. Fazel, Matrix Rank Minimization with Applications, 2002.

A. Flexer, D. Schnitzer, and J. Schlüter, A mirex metaanalysis of hubness in audio music similarity, Proc. of the 13th Int. Soc. for Music Information Retrieval Conf. (ISMIR, 2012.

E. Fournie, Un Test de type Kolmogorov-smirnov pour processus de diffusion ergodiques, 1992.
URL : https://hal.archives-ouvertes.fr/inria-00076932

M. Frank and P. Wolfe, An algorithm for quadratic programming, Naval Research Logistics Quarterly, vol.3, issue.1-2, pp.95-110, 1956.
DOI : 10.1002/nav.3800030109

J. H. Friedman, machine., The Annals of Statistics, vol.29, issue.5, pp.1189-1232, 2000.
DOI : 10.1214/aos/1013203451

A. M. Frieze, R. Kannan, and S. Vempala, Fast monte-carlo algorithms for finding low-rank approximations, 39th Annual Symposium on Foundations of Computer Science, FOCS '98, pp.370-378, 1998.

P. Geurts, D. Ernst, and L. Wehenkel, Extremely randomized trees, Machine Learning, vol.63, issue.1, pp.3-42, 2006.
DOI : 10.1007/s10994-006-6226-1
URL : https://hal.archives-ouvertes.fr/hal-00341932

S. Girgin, J. Mary, P. Preux, N. , and O. , Managing advertising campaigns -an approximate planning approach, Frontiers of Computer Science, 2011.
URL : https://hal.archives-ouvertes.fr/hal-00747722

T. Graepel, J. Q. Candela, T. Borchert, and R. Herbrich, Web-scale bayesian clickthrough rate prediction for sponsored search advertising in microsoftâs bing search engine, Proceedings of the 27th International Conference on Machine Learning ICML 2010, 2010.

O. Granmo, A Bayesian Learning Automaton for Solving Two-Armed Bernoulli Bandit Problems, Proc. of the 7 th Internaitonal Conference on Machine Learning and Applications (ICML-A), pp.23-30, 2008.

F. Guillou, R. Gaudel, J. Mary, and P. Preux, User Engagement as Evaluation, Proceedings of the 2014 Recommender Systems Challenge on, RecSysChallenge '14, 2014.
DOI : 10.1145/2668067.2668073
URL : https://hal.archives-ouvertes.fr/hal-01077986

R. A. Harshman and M. E. Lundy, PARAFAC: Parallel factor analysis, Computational Statistics & Data Analysis, vol.18, issue.1, pp.39-72, 1994.
DOI : 10.1016/0167-9473(94)90132-5

R. Herbrich, T. Minka, and T. Graepel, Trueskill TM : A bayesian skill rating system, Advances in Neural Information Processing Systems 19, pp.569-576, 2007.

Y. Hu, Y. Koren, and C. Volinsky, Collaborative Filtering for Implicit Feedback Datasets, 2008 Eighth IEEE International Conference on Data Mining, pp.263-272, 2008.
DOI : 10.1109/ICDM.2008.22

P. Jain and S. Oh, Provable tensor factorization with missing data, Advances in Neural Information Processing Systems 27, pp.1431-1439, 2014.

R. Jenatton, N. L. Roux, A. Bordes, and G. R. Obozinski, A latent factor model for highly multi-relational data, Advances in Neural Information Processing Systems 25, pp.3167-3175, 2012.
URL : https://hal.archives-ouvertes.fr/hal-00776335

C. Jose, P. Goyal, P. Aggrwal, and M. Varma, Local deep kernel learning for efficient non-linear svm prediction, Proceedings of the International Conference on Machine Learning, 2013.

S. M. Kakade, S. Shalev-shwartz, and A. Tewari, Efficient bandit algorithms for online multiclass prediction, Proceedings of the 25th international conference on Machine learning, ICML '08, pp.440-447, 2008.
DOI : 10.1145/1390156.1390212

R. Kannan and S. Vempala, Spectral algorithms. Foundations and Trends in Theoretical Computer Science, pp.3-4157, 2009.

L. V. Kantorovich and G. S. Rubinstein, On a function space in certain extremal problems, Dokl. Akad. Nauk USSR, vol.115, pp.1058-1061, 1957.

S. Karayev, A. Hertzmann, H. Winnemoeller, A. Agarwala, D. et al., Recognizing Image Style, Proceedings of the British Machine Vision Conference 2014, 2013.
DOI : 10.5244/C.28.122

A. Khaleghi, D. Ryabko, J. Mary, and P. Preux, Online Clustering of Processes, AISTATS 2012, pp.601-609, 2012.
URL : https://hal.archives-ouvertes.fr/hal-00765462

D. Kifer, S. Ben-david, and J. Gehrke, Detecting Change in Data Streams, Proceedings of the Thirtieth International Conference on Very Large Data Bases, pp.180-191, 2004.
DOI : 10.1016/B978-012088469-8.50019-X

A. N. Kolmogorov, Sulla determinazione empirica di una legge di distribuzione. Giornale dell'Istituto Italiano degli Attuari, pp.83-91, 1933.

V. Kuleshov, A. Chaganty, and P. Liang, Tensor factorization via matrix factorization, Artificial Intelligence and Statistics (AISTATS), 2015.

J. Langford, R. Oliveira, and B. Zadrozny, Predicting conditional quantiles via reduction to classification, UAI '06, Proceedings of the 22nd Conference in Uncertainty in Artificial Intelligence, 2006.

J. Langford, A. Strehl, and J. Wortman, Exploration scavenging, Proceedings of the 25th international conference on Machine learning, ICML '08, 2008.
DOI : 10.1145/1390156.1390223

J. Langford and T. Zhang, The Epoch-Greedy Algorithm for Multi-armed Bandits with Side Information, 20 th Advances in Neural Information Processing Systems (NIPS), pp.817-824, 2008.

L. D. Lathauwer, B. D. Moor, and J. Vandewalle, A Multilinear Singular Value Decomposition, SIAM Journal on Matrix Analysis and Applications, vol.21, issue.4, pp.1253-1278, 2000.
DOI : 10.1137/S0895479896305696

N. Lathia, S. Hailes, L. Capra, and X. Amatriain, Temporal diversity in recommender systems, Proceeding of the 33rd international ACM SIGIR conference on Research and development in information retrieval, SIGIR '10, pp.210-217, 2010.
DOI : 10.1145/1835449.1835486

Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, Gradient-based learning applied to document recognition, Proceedings of the IEEE, pp.2278-2324, 1998.
DOI : 10.1109/5.726791

B. Li, Q. Yang, and X. Xue, Can movies and books collaborate?: Cross-domain collaborative filtering for sparsity reduction, Proceedings of the 21st International Jont Conference on Artifical Intelligence, IJCAI'09, pp.2052-2057, 2009.

L. Li, W. Chu, J. Langford, and R. E. Schapire, A contextual-bandit approach to personalized news article recommendation, Proceedings of the 19th international conference on World wide web, WWW '10, pp.661-670, 2010.
DOI : 10.1145/1772690.1772758

L. Li, W. Chu, J. Langford, W. , and X. , Unbiased offline evaluation of contextualbandit-based news article recommendation algorithms, Proc. Web Search and Data Mining (WSDM), pp.297-306, 2011.

L. Li, R. Munos, and C. Szepesvári, Toward minimax off-policy value estimation, Proceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics, AISTATS 2015 JMLR Proceedings. JMLR.org, 2015.

M. Mahdian and H. Nazerzadeh, Allocating online advertisement space with unreliable estimates, Proceedings of the 8th ACM conference on Electronic commerce , EC '07, pp.288-294, 2007.
DOI : 10.1145/1250910.1250952

J. Mary, R. Gaudel, and P. Preux, Bandits Warm-up Cold Recommender Systems, 2014.
URL : https://hal.archives-ouvertes.fr/hal-01022628

J. Mary, O. Nicol, and P. Preux, Improving offline evaluation of contextual bandit algorithms via bootstrapping techniques, Proc. ICML, JMLR WCP, 2014.
URL : https://hal.archives-ouvertes.fr/hal-00990840

B. C. May, N. Korda, A. Lee, L. , and D. S. , Optimistic bayesian sampling in contextual-bandit problems, J. Mach. Learn. Res, vol.13, pp.2069-2106, 2012.

R. Mazumder, T. Hastie, and R. Tibshirani, Spectral regularization algorithms for learning large incomplete matrices, J. Mach. Learn. Res, vol.11, pp.2287-2322, 2010.

A. Mehta, A. Saberi, U. Vazirani, and V. Vazirani, AdWords and generalized online matching, Proc. of the 46 th Annual IEEE Symposium on Foundations of Computer Science (FOCS), pp.264-273, 2005.
DOI : 10.1145/1284320.1284321

G. A. Miller, WordNet: a lexical database for English, Communications of the ACM, vol.38, issue.11, pp.39-41, 1995.
DOI : 10.1145/219717.219748

O. Moreno, B. Shapira, L. Rokach, S. , and G. , TALMUD, Proceedings of the 21st ACM international conference on Information and knowledge management, CIKM '12, pp.425-434, 2012.
DOI : 10.1145/2396761.2396817

Y. Nesterov, Introductory lectures on convex optimization : a basic course Applied optimization, 2004.
DOI : 10.1007/978-1-4419-8853-9

O. Nicol, Data-driven evaluation of Contextual Bandit algorithms and applications to Dynamic Recommendation, 2014.
URL : https://hal.archives-ouvertes.fr/tel-01297407

O. Nicol, J. Mary, and P. Preux, Icml exploration and exploitation challenge: Keep it simple !, Journal of Machine Learning Research, 2012.

W. Pan, E. W. Xiang, N. N. Liu, and Q. Y. , Transfer learning in collaborative filtering for sparsity reduction, AAAI, 2010.

S. Pandey, D. Agarwal, D. Chakrabarti, and V. Josifovski, Bandits for Taxonomies: A Model-based Approach, Proc. of the 7 th SIAM International Conference on Data Mining, 2007.
DOI : 10.1137/1.9781611972771.20

S. Pandey and C. Olston, Handling Advertisements of Unknown Quality in Search Advertising, 18 th Advances in Neural Information Processing Systems (NIPS), pp.1065-1072, 2006.

D. Precup, R. S. Sutton, and S. P. Singh, Eligibility traces for off-policy policy evaluation, Proceedings of the Seventeenth International Conference on Machine Learning, pp.759-766, 2000.

J. Quionero-candela, M. Sugiyama, A. Schwaighofer, L. , and N. D. , Dataset Shift in Machine Learning, 2009.

S. Rendle, G. I. Webb, B. Liu, C. Zhang, D. Gunopulos et al., Factorization Machines, 2010 IEEE International Conference on Data Mining, pp.14-17, 2010.
DOI : 10.1109/ICDM.2010.127

S. Rendle, Factorization Machines with libFM, ACM Transactions on Intelligent Systems and Technology, vol.3, issue.3, pp.1-5722, 2012.
DOI : 10.1145/2168752.2168771

S. Rendle, C. Freudenthaler, and L. Schmidt-thieme, Factorizing personalized Markov chains for next-basket recommendation, Proceedings of the 19th international conference on World wide web, WWW '10, pp.811-820, 2010.
DOI : 10.1145/1772690.1772773

Y. Research, R6b -yahoo! front page today module user click log dataset, 2012.

D. Ryabko, Clustering processes, Proc. the 27th International Conference on Machine Learning, pp.919-926, 2010.
URL : https://hal.archives-ouvertes.fr/inria-00477238

D. Ryabko, Discrimination Between B-Processes is Impossible, Journal of Theoretical Probability, vol.44, issue.6, pp.565-575, 2010.
DOI : 10.1007/s10959-009-0263-1
URL : https://hal.archives-ouvertes.fr/hal-00639537

D. Ryabko, On the relation between realizable and non-realizable cases of the sequence prediction problem, Journal of Machine Learning Research, vol.12, pp.2161-2180, 2011.
URL : https://hal.archives-ouvertes.fr/hal-00639474

D. Ryabko and J. Mary, Reducing statistical time-series problems to binary classification, Neural Information Processing Systems (NIPS), 2012.
URL : https://hal.archives-ouvertes.fr/hal-00675637

D. Ryabko and J. Mary, A binary-classification-based metric between time-series distributions and its use in statistical and learning problems, Journal of Machine Learning Research, 2013.
URL : https://hal.archives-ouvertes.fr/hal-00913240

D. Ryabko and B. Ryabko, Nonparametric Statistical Inference for Ergodic Processes, IEEE Transactions on Information Theory, vol.56, issue.3, pp.1430-1435, 2010.
DOI : 10.1109/TIT.2009.2039169
URL : https://hal.archives-ouvertes.fr/inria-00269249

B. Ryako, Prediction of random sequences and universal coding. Problems of Information Transmission, pp.87-96, 1988.

A. Said, S. Dooms, B. Loni, and D. Tikk, Recommender systems challenge 2014, Proceedings of the 8th ACM Conference on Recommender systems, RecSys '14, 2014.
DOI : 10.1145/2645710.2645779

A. Said, S. Dooms, B. Loni, and D. Tikk, Recsys challenge, 2014.

R. Salakhutdinov and A. Mnih, Probabilistic matrix factorization, Advances in Neural Information Processing Systems Proceedings of the Twenty-First Annual Conference on Neural Information Processing Systems, pp.1257-1264, 2007.

B. Schölkopf and A. Smola, Learning with Kernels: Support Vector Machines, Regularization , Optimization, and Beyond. Adaptive computation and machine learning, 2002.

P. Serdyukov, G. Dupret, and N. Craswell, Yandex challenge, 2014.

P. Shivaswamy and T. Joachims, Online learning with preference feedback, NIPS workshop on choice models and preference learning, 2011.

R. Solomonoff, Complexity-based induction systems: Comparisons and convergence theorems. Information Theory, IEEE Transactions, vol.24, issue.4, pp.422-432, 1978.

R. Sutton and A. Barto, Reinforcement Learning: An Introduction. Adaptive Computation and Machine Learning Series, 1998.
DOI : 10.1007/978-1-4615-3618-5

M. Valko, N. Korda, R. Munos, I. N. Flaounas, C. et al., Finite-time analysis of kernelised contextual bandits, 2013.
URL : https://hal.archives-ouvertes.fr/hal-00826946

C. Wang, S. Kulkarni, and H. Poor, Bandit problems with side observations, IEEE Transactions on Automatic Control, vol.50, issue.3, pp.338-355, 2005.
DOI : 10.1109/TAC.2005.844079

Y. Wang, J. Yves-audibert, M. , and R. , Algorithms for infinitely many-armed bandits, Advances in Neural Information Processing Systems 21, pp.1729-1736, 2009.

K. Weinberger, A. Dasgupta, J. Langford, A. Smola, and J. Attenberg, Feature hashing for large scale multitask learning, Proceedings of the 26th Annual International Conference on Machine Learning, ICML '09, pp.1113-1120, 2009.
DOI : 10.1145/1553374.1553516

Y. Zhou, D. Wilkinson, R. Schreiber, and R. Pan, Large-Scale Parallel Collaborative Filtering for the Netflix Prize, Proceedings of the 4th international conference on Algorithmic Aspects in Information and Management (AAIM), pp.337-348, 2008.
DOI : 10.1007/978-3-540-68880-8_32

V. M. Zolotarev, Probability metrics. j-THEORY-PROBAB-APPL, pp.278-302, 1983.

D. Agarwal, . Chen, . Bee-chung, and P. Elango, Spatio-temporal models for estimating clickthrough rate, Proceedings of the 18th international conference on World wide web(WWW), pp. 21?30, 2009.

J. Audibert, . Munos, . Rémi, and C. Szepesvári, Exploration???exploitation tradeoff using variance estimates in multi-armed bandits, Theoretical Computer Science, vol.410, issue.19, pp.1876-1902, 2009.
DOI : 10.1016/j.tcs.2009.01.016
URL : https://hal.archives-ouvertes.fr/hal-00711069

P. Auer, N. Cesa-bianchi, and P. Fischer, Finite-time analysis of the multiarmed bandit problem, Machine Learning, vol.47, issue.2/3, pp.235-256, 2002.
DOI : 10.1023/A:1013689704352

C. M. Bishop, Neural Networks for Pattern Recognition Neural Networks for Pattern Recognition, 1995.

. Dudík, . Miroslav, J. Langford, and L. Li, Doubly robust policy evaluation and learning. CoRR, abs/1103, 2011.

B. Efron, Bootstrap methods: Another look at the jackknife . The Annals of Statistics, pp.1-26, 1979.

A. Kleiner, . Talwalkar, . Ameet, . Sarkar, . Purnamrita et al., The big data bootstrap, Proceedings of the 29th International Conference on Machine Learning (ICML- 12), ICML '12, pp.1759-1766, 2012.

R. Kohavi, . Longbotham, . Roger, . Sommerfield, . Dan et al., Controlled experiments on the web: survey and practical guide, Data Mining and Knowledge Discovery, vol.33, issue.6, pp.140-181, 2009.
DOI : 10.1007/s10618-008-0114-1

P. Koistinen and L. Holmström, Kernel regression and backpropagation training with noise, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks, pp.1033-1039, 1992.
DOI : 10.1109/IJCNN.1991.170429

J. Langford and T. Zhang, The epoch-greedy algorithm for multi-armed bandits with side information, Proc. NIPS, 2007.

J. Langford, A. Strehl, and J. Wortman, Exploration scavenging, Proceedings of the 25th international conference on Machine learning, ICML '08, pp.528-535, 2008.
DOI : 10.1145/1390156.1390223

K. Levchenko, Notes from a lecture of van vu at university of california, san diego, 2005.

L. Li, . Chu, . Wei, J. Langford, and R. E. Schapire, A contextual-bandit approach to personalized news article recommendation, Proceedings of the 19th international conference on World wide web, WWW '10, pp.661-670, 2010.
DOI : 10.1145/1772690.1772758

L. Li, . Chu, . Wei, J. Langford, and X. Wang, Unbiased offline evaluation of contextual-bandit-based news article recommendation algorithms, Proceedings of the fourth ACM international conference on Web search and data mining, WSDM '11, pp.297-306, 2011.
DOI : 10.1145/1935826.1935878

T. Lu, D. Pál, and M. Pál, Contextual multi-armed bandits, Proc. of the 13 th Artificial Intelligence and Statistics (AI & Stats), JMLR: W&CP 9, pp.13-15

H. Robbins, Some aspects of the sequential design of experiments, Bulletin of the American Mathematical Society, vol.58, issue.5, pp.527-535, 1952.
DOI : 10.1090/S0002-9904-1952-09620-8

D. W. Scott, Multivariate Density Estimation: Theory, Practice, and Visualization, 1992.
DOI : 10.1002/9781118575574

G. Shani and A. Gunawardana, Evaluating Recommendation Systems, Recommender systems handbook, pp.257-297, 2011.
DOI : 10.1007/978-0-387-85820-3_8

A. L. Strehl, . Langford, . John, . Li, . Lihong et al., Learning from logged implicit exploration data, Proc. NIPS, pp.2217-2225, 2010.

R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. Adaptive Computation and Machine Learning Series, 1998.
DOI : 10.1007/978-1-4615-3618-5

W. R. Thompson, On the likelihood that one unknown probability exceeds another in view of the evidence of two samples, Biometrika, vol.25, pp.3-4285, 1933.

P. Auer, P. Nicoì-o-cesa-bianchi, and . Fischer, Finite-time analysis of the multiarmed bandit problem, pp.235-256, 2002.

S. Girgin, J. Mary, . Ph, O. Preux, and . Nicol, Advertising campaigns man- agement: Should we be greedy?, The 10th IEEE International Conference on Data Mining (ICDM-2010), pp.821-826, 2010.

T. Graepel, J. Q. Candela, T. Borchert, and R. Herbrich, Web-scale Bayesian click-through rate prediction for sponsored search advertising in Microsoft's Bing search engine, Proceedings of the 27th International Conference on Machine Learning (ICML-2010), pp.13-20, 2010.

R. Herbrich, T. Minka, and T. Graepel, Trueskill tm : A bayesian skill rating system, Advances in Neural Information Processing Systems 19 (NIPS-2006), pp.569-576, 2007.

G. J. Mc-lachlan and T. Krishnan, The EM Algorithm and Extensions, 1996.

R. S. Sutton, Learning to predict by the methods of temporal differences, Machine Learning, vol.34, issue.1
DOI : 10.1007/BF00115009

P. Auer, N. Cesa-bianchi, and P. Fischer, Finite-time analysis of the multiarmed bandit problem, Machine Learning, vol.47, issue.2/3, pp.235-256, 2002.
DOI : 10.1023/A:1013689704352

N. Abe and A. Nakamura, Learning to optimally schedule Internet banner advertisements, Proceedings of the 16th International Conference on Machine Learning, pp.12-21, 1999.

O. Granmo, A Bayesian learning automaton for solving two-armed Bernoulli bandit problems, Proceedings of the 7th International Conference on Machine Learning and Applications, pp.23-30, 2008.

M. Langheinrich, A. Nakamura, N. Abe, T. Kamba, and Y. Koseki, Unintrusive customization techniques for Web advertising, Computer Networks, vol.31, issue.11-16, pp.31-42, 1999.
DOI : 10.1016/S1389-1286(99)00033-X

A. Nakamura and N. Abe, Improvements to the Linear Programming Based Scheduling of Web Advertisements, Electronic Commerce Research, vol.5, issue.1, pp.75-98, 2005.
DOI : 10.1023/B:ELEC.0000045974.88926.88

S. Pandey, D. Agarwal, D. Chakrabarti, and V. Josifovski, Bandits for Taxonomies: A Model-based Approach, Proceedings of the 7th SIAM International Conference on Data Mining, 2007.
DOI : 10.1137/1.9781611972771.20

J. Langford and T. Zhang, The epoch-greedy algorithm for multi-armed bandits with side information, Proceedings of 20th Advances in Neural Information Processing Systems, pp.817-824, 2008.

C. C. Wang, S. R. Kulkarni, S. Poor, and H. , Bandit problems with side observations, IEEE Transactions on Automatic Control, issue.3, pp.50-338, 2005.

W. Li, X. Wang, R. Zhang, Y. Cui, J. Mao et al., Exploitation and exploration in a performance based contextual advertising system, Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining, KDD '10, pp.27-36, 2010.
DOI : 10.1145/1835804.1835811

S. Pandey and C. Olston, Handling advertisements of unknown quality in search advertising, Proceedings of 18th Advances in Neural Information Processing Systems, pp.1065-1072, 2006.

D. Agarwal, B. Chen, and P. Elango, Explore/Exploit Schemes for Web Content Optimization, 2009 Ninth IEEE International Conference on Data Mining, pp.1-10, 2009.
DOI : 10.1109/ICDM.2009.52

L. Li, W. Chu, J. Langford, and R. Schapire, A contextual-bandit approach to personalized news article recommendation, Proceedings of the 19th international conference on World wide web, WWW '10, pp.661-670, 2010.
DOI : 10.1145/1772690.1772758

M. Richardson, E. Dominowska, and R. Ragno, Predicting clicks, Proceedings of the 16th international conference on World Wide Web , WWW '07, pp.521-530, 2007.
DOI : 10.1145/1242572.1242643

D. Agarwal, B. C. Chen, and P. Elango, Spatio-temporal models for estimating click-through rate, Proceedings of the 18th international conference on World wide web, WWW '09, pp.21-30, 2009.
DOI : 10.1145/1526709.1526713

D. Agarwal, A. Broder, D. Chakrabarti, D. Diklic, V. Josifovski et al., Estimating rates of rare events at multiple resolutions, Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining , KDD '07, pp.16-25, 2007.
DOI : 10.1145/1281192.1281198

X. Wang, W. Li, Y. Cui, B. Zhang, and J. Mao, Clickthrough rate estimation for rare events in online advertising, Online Multimedia Advertising: Techniques and Technologies. Hershey: IGI Global, 2010.

T. K. Fan and C. Chang, Sentiment-oriented contextual advertising, Knowledge and Information Systems, vol.20, issue.3, pp.321-344, 2010.
DOI : 10.1007/s10115-009-0222-2

A. Mehta, A. Saberi, U. Vazirani, and V. Vazirani, AdWords and Generalized On-line Matching, 46th Annual IEEE Symposium on Foundations of Computer Science (FOCS'05), pp.264-273, 2005.
DOI : 10.1109/SFCS.2005.12

J. Langford, A. Strehl, and J. Wortman, Exploration scavenging, Proceedings of the 25th international conference on Machine learning, ICML '08, pp.528-535, 2008.
DOI : 10.1145/1390156.1390223

F. R. Bach and M. I. Jordan, Learning Graphical Models for Stationary Time Series, IEEE Transactions on Signal Processing, vol.52, issue.8, pp.2189-2199, 2004.
DOI : 10.1109/TSP.2004.831032

M. F. Balcan, A. Blum, and S. Vempala, A discriminative framework for clustering via similarity functions, Proceedings of the fourtieth annual ACM symposium on Theory of computing, STOC 08, 2008.
DOI : 10.1145/1374376.1374474

C. Biernacki, G. Celeux, and G. Govaert, Assessing a mixture model for clustering with the integrated completed likelihood. Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol.22, issue.7, pp.719-725, 2000.

P. Billingsley, Statistical Methods in Markov Chains, The Annals of Mathematical Statistics, vol.32, issue.1, pp.12-40, 1961.
DOI : 10.1214/aoms/1177705136

P. Billingsley, Convergence of probability measures, 1999.
DOI : 10.1002/9780470316962

D. Bosq, Nonparametric Statistics for Stochastic Processes. Estimation and Prediction, 1996.

R. C. Bradley, Basic Properties of Strong Mixing Conditions. A Survey and Some Open Questions, Probability Surveys, pp.107-144, 2005.
DOI : 10.1214/154957805100000104

E. Carlstein and S. Lele, Nonparametric Change-Point Estimation for Data from an Ergodic Sequence, Theory of Probability & Its Applications, vol.38, issue.4, pp.910-917, 1993.
DOI : 10.1137/1138073

R. Cilibrasi and P. M. Vitanyi, Clustering by Compression, IEEE Transactions on Information Theory, vol.51, issue.4, pp.1523-1545, 2005.
DOI : 10.1109/TIT.2005.844059

P. Doukhan and G. Lang, Donatas Surgailis, and GillesTeyssì ere. Dependence in Probability and Statistics, 2010.

R. Grossi and J. S. Vitter, Compressed Suffix Arrays and Suffix Trees with Applications to Text Indexing and String Matching, SIAM Journal on Computing, vol.35, issue.2, pp.378-407, 2005.
DOI : 10.1137/S0097539702402354

M. Gutman, Asymptotically optimal classification for multiple tests with empirically observed statistics, IEEE Transactions on Information Theory, vol.35, issue.2, pp.402-408, 1989.
DOI : 10.1109/18.32134

T. Jebara, Y. Song, and K. Thadani, Spectral Clustering and Embedding with Hidden Markov Models, Machine Learning: ECML 2007, pp.164-175, 2007.
DOI : 10.1007/978-3-540-74958-5_18

I. Katsavounidis, C. Kuo, and Z. Zhang, A new initialization technique for generalized Lloyd iteration, IEEE Signal Processing Letters, vol.1, issue.10, pp.144-146, 1994.
DOI : 10.1109/97.329844

A. Khaleghi and D. Ryabko, Locating changes in highly-dependent data with unknown number of change points, Neural Information Processing Systems (NIPS), Lake Tahoe, Nevada, United States, 2012.
URL : https://hal.archives-ouvertes.fr/hal-00765436

A. Khaleghi, D. Ryabko, J. Mary, and P. Preux, Online clustering of processes, AISTATS, JMLR W&CP 22, pp.601-609, 2012.
URL : https://hal.archives-ouvertes.fr/hal-00765462

A. Khaleghi and D. Ryabko, Asymptotically consistent estimation of the number of change points in highly dependent time series, ICML, JMLR W&CP, pp.539-547, 2014.
URL : https://hal.archives-ouvertes.fr/hal-01026583

J. Kleinberg, An impossibility theorem for clustering, 15th Conf. Neiral Information Processing Systems (NIPS'02), pp.446-453, 2002.

I. Kontoyiannis and Y. M. Suhov, Prefixes and the entropy rate for long-range sources, Proceedings of 1994 IEEE International Symposium on Information Theory, pp.194-194, 1994.
DOI : 10.1109/ISIT.1994.394774

M. Kumar, N. R. Patel, and J. Woo, Clustering seasonality patterns in the presence of errors, Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining , KDD '02, pp.557-563, 2002.
DOI : 10.1145/775047.775129

E. Lehmann, Testing Statistical Hypotheses, 1986.

L. Li and B. A. Prakash, Time series clustering: Complex is simpler!, 2011.

M. Mahajan, P. Nimbhorkar, and K. Varadarajan, The Planar k-Means Problem is NP-Hard, WALCOM '09: Proceedings of the 3rd International Workshop on Algorithms and Computation, pp.274-285, 2009.
DOI : 10.1109/TC.1981.6312176

G. Morvai and B. Weiss, On classifying processes, Bernoulli, vol.11, issue.3, pp.523-532, 2005.
DOI : 10.3150/bj/1120591187

G. Morvai and B. Weiss, A note on prediction for discrete time series, Kybernetika, vol.48, issue.4, pp.809-823, 2012.

D. S. Ornstein and B. Weiss, How Sampling Reveals a Process, The Annals of Probability, vol.18, issue.3, pp.905-930, 1990.
DOI : 10.1214/aop/1176990729

E. Rio, Théorie asymptotique des processus aléatoires faiblement dépendants, 1999.

B. Ryabko, Prediction of random sequences and universal coding. Problems of Information Transmission, pp.87-96, 1988.

B. Ryabko and J. Astola, Universal codes as a basis for time series testing, Statistical Methodology, vol.3, issue.4, pp.375-397, 2006.
DOI : 10.1016/j.stamet.2005.10.004

B. Ryabko, Applications of universal source coding to statistical analysis of time series. Selected Topics in Information and Coding Theory, pp.289-338, 2010.

D. Ryabko, Clustering processes, Proc. the 27th International Conference on Machine Learning (ICML 2010), pp.919-926, 2010.
URL : https://hal.archives-ouvertes.fr/inria-00477238

D. Ryabko, Testing composite hypotheses about discrete ergodic processes, TEST, vol.56, issue.3, pp.317-329, 2012.
DOI : 10.1007/s11749-011-0245-3
URL : https://hal.archives-ouvertes.fr/hal-00639477

D. Ryabko, Uniform hypothesis testing for finite-valued stationary processes, Statistics, vol.22, issue.1, pp.121-128, 2014.
DOI : 10.1007/s10959-009-0263-1
URL : https://hal.archives-ouvertes.fr/inria-00610009

D. Ryabko and J. Mary, Reducing statistical time-series problems to binary classification, Advances in Neural Information Processing Systems 25, pp.2069-2077, 2012.
URL : https://hal.archives-ouvertes.fr/hal-00675637

P. Shields, The Ergodic Theory of Discrete Sample Paths, 1996.
DOI : 10.1090/gsm/013

P. Smyth, Clustering sequences with hidden Markov models, Advances in Neural Information Processing Systems, pp.648-654, 1997.

E. Ukkonen, On-line construction of suffix trees, Algorithmica, vol.10, issue.3, pp.249-260, 1995.
DOI : 10.1007/BF01206331

S. Zhong and J. Ghosh, A unified framework for model-based clustering, Journal of Machine Learning Research, vol.4, pp.1001-1037, 2003.

T. M. Adams and A. B. , Uniform approximation of Vapnik???Chervonenkis classes, Bernoulli, vol.18, issue.4, pp.1310-1319, 2012.
DOI : 10.3150/11-BEJ379

M. Balcan, N. Bansal, A. Beygelzimer, D. Coppersmith, J. Langford et al., Robust Reductions from Ranking to Classification, Learning Theory, pp.604-619, 2007.
DOI : 10.1007/978-3-540-72927-3_43

M. Balcan, A. Blum, and S. Vempala, A discriminative framework for clustering via similarity functions, Proceedings of the fourtieth annual ACM symposium on Theory of computing, STOC 08, pp.671-680, 2008.
DOI : 10.1145/1374376.1374474

P. Billingsley, Ergodic Theory and Information, 1965.

D. Bosq, Nonparametric Statistics for Stochastic Processes. Estimation and Prediction, 1996.

. Ch.-ch, C. Chang, and . Lin, LIBSVM: A library for support vector machines, ACM Transactions on Intelligent Systems and Technology, vol.2, pp.27-28, 2011.

C. Cortes and V. Vapnik, Support-vector networks, Machine Learning, pp.273-297, 1995.
DOI : 10.1007/BF00994018

R. Fortet and E. Mourier, Convergence de la répartition empirique vers la répartition théoretique, Ann. Sci. Ec. Norm. Super., III. Ser, vol.70, issue.3, pp.267-285, 1953.

Z. Harchaoui, F. Bach, and E. Moulines, Kernel change-point analysis, Advances in Neural Information Processing Systems 21, pp.609-616, 2008.

L. V. Kantorovich and G. S. Rubinstein, On a function space in certain extremal problems, Dokl. Akad. Nauk USSR, vol.115, issue.6, pp.1058-1061, 1957.

R. L. Karandikar and M. Vidyasagar, Rates of uniform convergence of empirical means with mixing processes, Statistics & Probability Letters, vol.58, issue.3, pp.297-307, 2002.
DOI : 10.1016/S0167-7152(02)00124-4

A. Khaleghi, D. Ryabko, J. Mary, and P. Preux, Online clustering of processes, AISTATS, JMLR W&CP 22, pp.601-609, 2012.
URL : https://hal.archives-ouvertes.fr/hal-00765462

A. Khaleghi and D. Ryabko, Locating changes in highly dependent data with unknown number of change points, Advances in Neural Information Processing Systems 25, pp.3095-3103, 2012.
URL : https://hal.archives-ouvertes.fr/hal-00765436

A. Khaleghi and D. Ryabko, Nonparametric multiple change point estimation in highly dependent time series, Proc. 24th International Conf. on Algorithmic Learning Theory (ALT'13), Singapre, 2013.
URL : https://hal.archives-ouvertes.fr/hal-00913250

D. Kifer, . Sh, J. Ben-david, and . Gehrke, Detecting Change in Data Streams, Proc. the Thirtieth International Conference on Very Large Data Bases, pp.180-191, 2004.
DOI : 10.1016/B978-012088469-8.50019-X

A. N. Kolmogorov, Sulla determinazione empirica di una legge di distribuzione, G. Inst. Ital. Attuari, pp.83-91, 1933.

J. Langford, R. Oliveira, and B. Zadrozny, Predicting conditional quantiles via reduction to classification, Proc. of the 22th Conference on Uncertainty in Artificial Intelligence (UAI), 2006.

J. and R. Millán, On the need for on-line learning in brain-computer interfaces, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541), 2004.
DOI : 10.1109/IJCNN.2004.1381116

D. S. Ornstein and B. Weiss, How Sampling Reveals a Process, The Annals of Probability, vol.18, issue.3, pp.905-930, 1990.
DOI : 10.1214/aop/1176990729

D. Pollard, Convergence of Stochastic Processes, 1984.
DOI : 10.1007/978-1-4612-5254-2

B. Ryabko, Prediction of random sequences and universal coding. Problems of Information Transmission, pp.87-96, 1988.

B. Ryabko, Compression-Based Methods for Nonparametric Prediction and Estimation of Some Characteristics of Time Series, IEEE Transactions on Information Theory, vol.55, issue.9, pp.4309-4315, 2009.
DOI : 10.1109/TIT.2009.2025546

D. Ryabko, Clustering processes, Proc. the 27th International Conference on Machine Learning (ICML 2010), pp.919-926, 2010.
URL : https://hal.archives-ouvertes.fr/inria-00477238

D. Ryabko, Testing composite hypotheses about discrete ergodic processes, TEST, vol.56, issue.3, pp.317-329, 2012.
DOI : 10.1007/s11749-011-0245-3
URL : https://hal.archives-ouvertes.fr/hal-00639477

H. Sakoe and S. Chiba, Dynamic programming algorithm optimization for spoken word recognition, IEEE Transactions on Acoustics, Speech, and Signal Processing, vol.26, issue.1, pp.43-49, 1978.
DOI : 10.1109/TASSP.1978.1163055

P. Shields, The Ergodic Theory of Discrete Sample Paths, 1996.
DOI : 10.1090/gsm/013

R. J. Solomonoff, Complexity-based induction systems: Comparisons and convergence theorems, IEEE Transactions on Information Theory, vol.24, issue.4, pp.24422-432, 1978.
DOI : 10.1109/TIT.1978.1055913

V. M. Zolotarev, Probability metrics. Theory of Probability and Its Applications, pp.264-287, 1983.