, ProxyComponent/Order"/> 4 <service name="Notify" promote="MonitoringComponent/Notify"/> 5 <reference name="Fetch1" promote="Catalog/Fetch1"/> 6 <reference name="Fetch2" promote="Catalog/Fetch2"/> 7 <reference name="Fetch3" promote="Catalog/Fetch3"/> 8 <reference name="Order" promote="Catalog/Order"/> 9 <reference name="UsersInformation" promote="Catalog/UsersInformation"/> 10 11 <component name="ProxyComponent"> 12 <service name=

, Consult" target="Catalog/Consult"/> 18 <reference name="Order" target="Catalog/Order"/> 19 <reference name="Notify" target="MonitoringComponent/Notify"/> 20 <implementation, </service> 17 <reference name=

. , 84 6.3 Evaluation of the Framework Approach for Building Self-Adaptive Component-based Applications, </component> Evaluation and Validation Contents 6.1 Introduction

. , 89 6.4.2 Reinforcement Learning Parameters Selection for the Learning Efficiency, Evaluation of the Self-adaptive Decision Making Process approach based on Multi-step Reinforcement learning

. .. Synthesis,

. .. Conclusion, 98 this chapter to the evaluation and validation aspects of our contributions presented in Chapters 4 and 5. We make use of the Products Composite application case study, previously described in Section 5.4, to implement and run the experiments. We start by providing implementation details of our contributions in Section 6.2. Afterwards, we introduce the performed evaluations in two steps. In the first step in Section 6.3.1, 7.1 Fulfillment of Contributions Objectives

N. Belhaj, D. Belaïd, and H. Mukhtar, Framework for Building Self-Adaptive Component Applications based on Reinforcement Learning, IEEE International Conference on Services Computing (SCC), pp.17-24, 2018.
URL : https://hal.archives-ouvertes.fr/hal-01975700

N. Belhaj, D. Belaïd, and H. Mukhtar, Self-Adaptive Decision Making for the Management of Component-based Applications, On The Move, 25th International Conference on Cooperative Information Systems (CoopIS), pp.570-588, 2017.
URL : https://hal.archives-ouvertes.fr/hal-01681177

, Collaborative Autonomic Management of Distributed Component-based Applications, On The Move, 23rd International Conference on Cooperative Information Systems (CoopIS), pp.3-18, 2015.

, Collaborative Autonomic Container for the Management of Component-based Applications, IEEE 24th International Conference on Enabling Technologies: Infrastructures for Collaborative Enterprises (WETICE), pp.41-43, 2015.

. , Service Component Architecture Assembly Model Specification Version 1.1, OASIS Standard, p.13, 2009.

P. Horn, Autonomic Computing: IBM's Perspective on the State of Information Technology, xv, vol.14, p.15, 2001.

O. Sigaud and O. Buffet, Markov Decision Processes in Artificial Intelligence, vol.16, p.90, 2010.
URL : https://hal.archives-ouvertes.fr/inria-00432735

R. S. Sutton and A. G. Barto, Reinforcement Learning : An Introduction, xv, vol.18, p.19, 2012.

J. Soldani, T. Binz, U. Breitenbücher, F. Leymann, and A. Brogi, ToscaMart: A method for adapting and reusing cloud applications, Journal of Systems and Software, vol.113, p.23, 2016.

, An Architectural Blueprint for Autonomic Computing, vol.14, p.24

J. Adamczyk, R. Chojnacki, M. Jarzab, and K. Zielinski, Rule Engine Based Lightweight Framework for Adaptive and Autonomic Computing, vol.38, p.40, 2008.

C. Ruz, F. Baude, and B. Sauvan, Component-based Generic Approach for Reconfigurable Management of Component-based SOA Applications, 3rd International Workshop on Monitoring, Adaptation and Beyond, vol.38, p.40, 2010.

M. B. Alaya and T. Monteil, FRAMESELF: A Generic Context-Aware Autonomic Framework for SelfManagement of Distributed Systems," in WETICE, xv, vol.5, p.40, 2012.

S. Frey, A. Diaconescu, D. Menga, and I. M. Demeure, Towards a Generic Architecture and Methodology for Multi-goal, Highly-distributed and Dynamic Autonomic Systems, 10th International Conference on Autonomic Computing, ICAC'13, vol.38, p.40, 2013.

F. Paraiso, P. Merle, and L. Seinturier, soCloud: a service-oriented component-based PaaS for managing portability, provisioning, elasticity, and high availability across multiple clouds, Computing, vol.98, issue.5, p.40, 2016.
URL : https://hal.archives-ouvertes.fr/hal-01019385

L. Beishui, Y. Yuan, and G. Ji, Conceptual Model and Realization Methods of Autonomic Computing, Journal of Software, vol.19, p.30, 2008.

S. M. Gueye, N. D. Palma, and E. Rutten, Component-Based Autonomic Managers for Coordination Control, COORDINATION, 2013. xv, 5, 31, vol.32, p.40
URL : https://hal.archives-ouvertes.fr/hal-00949562

J. Cano, G. Delaval, and E. Rutten, Coordination of ECA rules by verification and control, COORDINATION, 2014. xv, 5, vol.32, p.40
URL : https://hal.archives-ouvertes.fr/hal-01006186

G. Tesauro, Reinforcement Learning in Autonomic Computing: A Manifesto and Case Studies, IEEE Internet Computing, vol.38, p.40, 2007.

M. Amoui, M. Salehie, S. Mirarab, and L. Tahvildari, Adaptive Action Selection in Autonomic Software Using Reinforcement Learning, Fourth International Conference on Autonomic and Autonomous Systems, vol.6, p.40, 2008.

P. Nawrocki and B. Sniezynski, Adaptive Service Management in Mobile Cloud Computing by Means of Supervised and Reinforcement Learning, Journal of Network and Systems Management, vol.26, issue.1, p.40, 2018.

M. P. Papazoglou, Service-oriented computing: concepts, characteristics and directions, Proceedings of the Fourth International Conference on Web Information Systems Engineering, vol.1, p.12, 2003.

, An Introduction to Creating Service Component Architecture Applications in Rational Application Developer Version 8.0, vol.60, p.61, 2010.

J. O. Kephart and D. M. Chess, The Vision of Autonomic Computing, vol.44, p.45, 2003.

G. Tesauro, N. K. Jong, R. Das, and M. N. Bennani, A Hybrid Reinforcement Learning Approach to Autonomic Resource Allocation, Proceedings of the 2006 IEEE International Conference on Autonomic Computing, ser. ICAC '06, vol.35, p.44, 2006.

M. R. Nami and K. Bertels, A survey of autonomic computing systems, The 3rd International Conference on Autonomic and Autonomous Systems, vol.3, p.44, 2007.

R. De-lemos, H. Giese, H. A. Müller, M. Shaw, J. Andersson et al., Software engineering for self-adaptive systems: A second research roadmap, p.44, 2013.
URL : https://hal.archives-ouvertes.fr/inria-00638157

L. P. Kaelbling, M. L. Littman, and A. P. Moore, Reinforcement Learning: A Survey, Journal of Artificial Intelligence Research, vol.15, issue.3, p.44, 1996.

M. L. Puterman, Markov Decision Processes: discrete stochastic dynamic programming, vol.44, p.52, 1994.

S. M. Lavalle, Sequential Decision Theory, Planning Algorithms, vol.4, p.44, 2006.

R. S. Sutton, Learning to predict by the methods of temporal differences, MACHINE LEARNING, vol.4, p.54, 1988.

. Oracle, Java Management Extensions Specification, version 1.4, vol.37, p.40, 2006.

A. Brogi, A. Canciani, J. Soldani, and P. Wang, Modelling the Behaviour of Management Operations in Cloud-based Applications, PNSE @ Petri Nets, ser. CEUR Workshop Proceedings, vol.1372, p.40, 2015.

R. Agrawal, M. Behrens, B. R. Byreddy, and M. Carlson, Cloud Application Management for Platforms Version 1.2, vol.37, p.40, 2017.

M. B. Alaya and T. Monteil, FRAMESELF: an ontology-based framework for the self-management of machine-to-machine systems, Concurrency and Computation: Practice and Experience, vol.27, issue.6, p.40, 2015.

B. Jacob, R. Lanyon-hogg, D. K. Nadgir, and A. F. Yassin, A Practical Guide to the IBM Autonomic Computing Toolkit RedBook, vol.38, p.40, 2004.

. , Topology and Orchestration Specification for Cloud Applications Version 1.0, vol.22, p.40, 2013.

S. Frey, A. Diaconescu, and I. Demeure, Architectural integration patterns for autonomic management systems, 9th IEEE International Conference and Workshops on the Engineering of Autonomic and Autonomous Systems (EASe), vol.38, p.40, 2012.

R. Raghavendra, P. Ranganathan, V. Talwar, Z. Wang, and X. Zhu, No power Struggles: Coordinated Multi-level Power Management for the Data Center, ASPLOS'13, vol.30, p.40, 2008.

S. Lightstone, M. Surendra, Y. Diao, S. S. Parekh, J. L. Hellerstein et al., Control Theory: a Foundational Technique for Self Managing Databases," in ICDE Workshops, vol.5, p.40, 2007.

Y. Diao, J. L. Hellerstein, S. Parekh, R. Griffith, G. Kaiser et al., A Control Theory Foundation for Self-managing Computing Systems, 12th IEEE International Conference and Workshops on the Engineering of Computer-Based Systems, vol.5, p.40, 2005.

R. Abid, G. Salaün, and N. D. Palma, Asynchronous synthesis techniques for coordinating autonomic managers in the cloud, Sci. Comput. Program, vol.146, p.40, 2017.
URL : https://hal.archives-ouvertes.fr/hal-01630717

S. M. Gueye, N. D. Palma, and E. Rutten, Coordinating energy-aware administration loops using discrete control, The Eighth International Conference on Autonomic and Autonomous Systems, vol.5, p.40, 2012.

H. Wang, X. Zhou, X. Zhou, W. Liu, W. Li et al., Adaptive Service Composition Based on Reinforcement Learning, ICSOC, vol.34, p.40, 2010.

H. Wang, X. Wang, X. Hu, X. Zhang, and M. Gu, A multi-agent reinforcement learning approach to dynamic service composition, Inf. Sci, vol.363, issue.6, p.40, 2016.

G. Tesauro, Online Resource Allocation Using Decompositional Reinforcement Learning, Proceedings, AAAI'05, vol.34, p.40, 2005.

J. Rao, X. Bu, K. Wang, and C. Xu, Self-adaptive Provisioning of Virtualized Resources in Cloud Computing, SIGMETRICS'11, vol.38, p.40, 2011.

O. Sigaud, O. Buffet, and A. E. , Processus décisionnels de Markov en intelligence artificielle, vol.52, p.89, 2008.

D. Booth, H. Haas, F. Mccabe, E. Newcomer, M. Champion et al., W3C Working Group Note, vol.11, p.12, 2004.

S. Burbeck, The Tao of e-business services: The evolution of Web applications into serviceoriented components with Web services, IBM Software Group, 2000.

I. and I. Websphere, Application V7 Feature Pack for Service Component Architecture. IBM, 2008.

. Available,

M. C. Huebscher and J. A. Mccann, A Survey of Autonomic Computing-Degrees, Models, and Applications, p.14, 2008.

S. J. Russell and P. Norvig, Artificial Intelligence: A Modern Approach, vol.15, 2003.

R. S. Sutton and A. G. Barto, Reinforcement Learning : An Introduction. A Bradford Book, vol.17, p.52, 2017.

P. Gerard, Apprentissage par Renforcement: Apprentissage Numérique, p.17, 2008.

. , Web Services Business Process Execution Language Version 2.0, vol.23, p.48, 2007.

. Omg, Business Process Model and Notation (BPMN) Version 2.0, p.23, 2011.

T. Binz, U. Breitenbücher, F. Haupt, O. Kopp, F. Leymann et al., OpenTOSCA-A Runtime for TOSCA-based Cloud Applications, 11 th International Conference on Service-Oriented Computing, ser, 2013.

S. Bechhofer, F. Van-harmelen, J. Hendler, I. Horrocks, D. Mcguinness et al., OWL Web Ontology Language Reference, World Wide Web Consortium (W3C), Recommendation, 2004.

, ETSI ETSI TS 102 690 V1.1.1: Machine-to-Machine communications (M2M); Functional architecture (2011-10)

B. Michelson, Event-Driven Architecture Overview, p.27, 2006.

S. Kanaparti, A Distributed-SOA Model for Unified Communication Services, ICWS. IEEE Computer Society, vol.27, pp.529-536, 2008.

U. Brandes, M. Eiglsperger, I. Herman, M. Himsolt, and M. S. Marshall, GraphML Progress Report: Structural Layer Proposal, Proceedings of the 9th International Symposium Graph Drawing (GD '01) LNCS 2265, pp.501-512, 2002.

H. Chen and C. Cao, Research and Application of Distributed OSGi for Cloud Computing, International Conference on Computational Intelligence and Software Engineering (CiSE), vol.27, pp.1-5, 2010.

A. Mcewen and H. Cassimally, Designing the Internet of Things, p.27, 2013.

W. L. Brogan, Modern Control Theory, p.29, 1991.

G. F. Franklin, D. J. Powell, and A. Emami-naeini, Feedback Control of Dynamic Systems, p.29, 2001.

S. Martello, D. Pisinger, and D. Vigo, The Three-Dimensional Bin Packing Problem, Oper. Res, vol.48, issue.2, p.31, 2000.

A. Filieri, M. Maggio, K. Angelopoulos, N. D&apos;ippolito, I. Gerostathopoulos et al., Control Strategies for Self-Adaptive Software Systems, ACM Trans. Auton. Adapt. Syst, vol.11, issue.4, pp.1-24, 2017.

, , vol.38, p.40

Y. Diao, J. L. Hellerstein, S. Parekh, R. Griffith, G. Kaiser et al., Self-managing systems: A control theory foundation, ECBS Workshop, pp.441-448, 2005.

C. G. Cassandras and S. Lafortune, Introduction to Discrete Event Systems, p.31, 2006.

. Inria-institute, /. Heptagon, and . Manual, , 2013.

H. Marchand, P. Bournai, M. L. Borgne, and P. L. Guernic, Synthesis of Discrete-Event Controllers based on the Signal Environment, Discrete Event Dynamic System: Theory and Applications, vol.10, issue.4, p.33, 2000.
URL : https://hal.archives-ouvertes.fr/hal-00546147

. Inria-institute, Reference Manual of the LNT to LOTOS Translator, (Formerly: Reference Manual of the LOTOS NT to LOTOS Translator), 2018.

H. Garavel, F. Lang, and R. Mateescu, Compositional verification of asynchronous concurrent systems using CADP, Acta Inf, vol.52, issue.4-5, pp.337-392, 2015.
URL : https://hal.archives-ouvertes.fr/hal-01247507

S. Haykin, Neural Networks: A Comprehensive Foundation, p.35, 1998.

N. Fernando, S. Loke, and W. Rahayu, Mobile cloud computing: A Survey, Future Gener. Comput. Syst, p.36, 2013.

S. B. Kotsiantis, Supervised Machine Learning: A Review of Classification Techniques, Proceedings of the 2007 Conference on Emerging Artificial Intelligence Applications in Computer Engineering: Real Word AI Systems with Applications in eHealth, HCI, Information Retrieval and Pervasive Technologies, pp.3-24, 2007.

P. Bianco, G. Lewis, and P. Merson, Service Level Agreements in Service-Oriented Architecture Environments, Tech. Rep, p.44, 2008.

P. Lalanda, J. A. Mccann, and A. Diaconescu, Autonomic Computing-Principles, Design and Implementation, ser. Undergraduate Topics in Computer Science, p.45, 2013.

R. Bellman, Dynamic Programming, p.53, 1957.

R. Bellman and S. Dreyfus, Functional Approximation and Dynamic Programming, Math. Tables and other Aids Comp, vol.13, p.53, 1959.

C. J. Watkins and P. Dayan, Q-learning, Machine Learning, vol.57, pp.279-292, 1992.

P. K. Mckinley, S. M. Sadjadi, H. C. Betty, E. P. Cheng, and . Kasten, Composing Adaptive Software, p.61, 2004.

L. Seinturier, P. Merle, D. Fournier, N. Dolet, V. Schiavoni et al., Reconfigurable SCA Applications with the FraSCAti Platform, Proceedings of IEEE International Conference on Services Computing., ser. SCC'09, vol.77, pp.268-275, 2009.
URL : https://hal.archives-ouvertes.fr/inria-00397856

S. Pat, Oracle SCA-The Power of the Composite, vol.84, 2009.

Z. Li and M. Parashar, A Decentralized Agent Framework for Dynamic Composition and Coordination for Autonomic Applications, DEXA Workshops, p.102, 2005.

R. G. Smith, The Contract Net Protocol: High-Level Communication and Control in a Distributed Problem Solver, IEEE Trans. Comput, vol.29, issue.12, pp.1104-1113, 1980.

O. Mola and M. Bauer, Collaborative Policy-based Autonomic Management: In a Hierarchical Model, CNSM, vol.102, pp.1-5, 2011.

, Towards Cloud Management by Autonomic Manager Collaboration, International Journal of Communications, vol.102, pp.790-802, 2011.

J. F. Oliveira, R. Sharrock, and T. Ledoux, Synchronization of Multiple Autonomic Control Loops: Application to Cloud Computing, Coordination Models and Languages-14th International Conference, pp.29-43, 2012.
URL : https://hal.archives-ouvertes.fr/hal-00682914

J. F. Oliveira and R. Sharrock, A Framework for the Coordination of Multiple Autonomic Managers in Cloud Environments, 7th IEEE International Conference on Self-Adaptive and SelfOrganizing Systems, SASO, pp.179-188, 2013.
URL : https://hal.archives-ouvertes.fr/hal-00839090

M. Mbaye and F. Krief, A Collaborative Knowledge Plane for Autonomic Networks, Autonomic Communication, pp.69-92, 2010.
URL : https://hal.archives-ouvertes.fr/hal-00411006

S. Muggleton and L. D. Raedt, Inductive Logic Programming: Theory and Methods, JOURNAL OF LOGIC PROGRAMMING, p.103, 1994.