thick line) ±2 v Dist (x) (thin lines) At the redundant point x = 2, the outputs are 1.5, 4, 7 and 7.5. The mean of the distribution-wise GP passes through the average of outputs. Contrarily to PI (cf. Figure 3.2), distribution-wise GP preserves the empirical variance: the kriging variance at x = 2 is equal to s 2 x=2 = 5, 87. . . . . . . . . . . . . 64 LIST OF FIGURES ,
) versus a GP model regularized by nugget (dashed lines) At x = 1, the number of repeated points is 3 (left) and is 100 (right). v N ug (x = 1) (thin dashed lines) shrinks as the number of repeated points increases while v Dist (x = 1) remains constant, p.66 ,
Ackley function (right) in dimension 5. The crosses show the time (number of calls) that the algorithm switchs from EGO to 83 LIST OF FIGURES 6.1 Effect of DoE and length-scale on EI function. The function to be optimized is the Sphere whose global minimum is located at 2.5. The blue and magenta curves represent the EI of kriging models with length-scales equal to 5 and 0.2, respectively. The crosses indicate the location of design points. The other parameters are fixed. The location of the third sample point changes from 2 to 1, p.110 ,
The effect of the nugget on Gaussian process emulators of computer models, Computational Statistics & Data Analysis, vol.56, issue.12, pp.4215-4228, 2012. ,
A Restart CMA Evolution Strategy With Increasing Population Size, 2005 IEEE Congress on Evolutionary Computation, pp.1769-1776, 2005. ,
DOI : 10.1109/CEC.2005.1554902
Analysis of Generalized Pattern Searches, SIAM Journal on Optimization, vol.13, issue.3, pp.889-903, 2002. ,
DOI : 10.1137/S1052623400378742
Theory of reproducing kernels. Transactions of the, 1950. ,
Ensemble of metamodels with optimized weight factors. Structural and Multidisciplinary Optimization, pp.279-294, 2009. ,
Cross Validation and Maximum Likelihood estimations of hyper-parameters of Gaussian processes with model misspecification, Computational Statistics & Data Analysis, vol.66, pp.55-69, 2013. ,
DOI : 10.1016/j.csda.2013.03.016
Robust Gaussian Process-Based Global Optimization Using a Fully Bayesian Expected Improvement Criterion, Learning and Intelligent Optimization, pp.176-190, 2011. ,
DOI : 10.1002/qre.945
URL : https://hal.archives-ouvertes.fr/hal-00607816
A tutorial on bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning, 2009. ,
A rigorous framework for optimization of expensive functions by surrogates, Structural Optimization, issue.17, pp.171-184, 1998. ,
Conditioning Diagnostics: Collinearity and Weak Data in Regression Wiley series in probability and mathematical statistics, 1991. ,
Sequential design of computer experiments for the estimation of a probability of failure, Statistics and Computing, vol.34, issue.4, pp.773-793, 2012. ,
DOI : 10.2307/1269548
URL : https://hal.archives-ouvertes.fr/hal-00689580
On Iterative Computation of Generalized Inverses and Associated Projections, SIAM Journal on Numerical Analysis, vol.3, issue.3, pp.410-419, 1966. ,
DOI : 10.1137/0703035
Neural Networks for Pattern Recognition, 1995. ,
Accelerating Evolutionary Algorithms With Gaussian Process Fitness Function Models, IEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews), vol.35, issue.2, pp.183-194, 2004. ,
DOI : 10.1109/TSMCC.2004.841917
Convex Optimization, 2004. ,
A stopping criterion for surrogate based optimization using ego, 10th World Congress on Structural and Multidisciplinary Optimization, 2013. ,
Efficient Global Optimization with Adaptive Target Setting, AIAA Journal, vol.52, issue.7, pp.1573-1578, 2014. ,
DOI : 10.1023/A:1011255519438
Fixed rank kriging for very large spatial data sets, Journal of the Royal Statistical Society: Series B (Statistical Methodology), vol.100, issue.1, pp.209-226, 2008. ,
DOI : 10.1137/1.9781611970128
Estimating Feasibility Using Multiple Surrogates and ROC Curves, 54th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, 2013. ,
DOI : 10.1214/ss/1030037906
URL : https://hal.archives-ouvertes.fr/emse-00806068
Statistics for spatial data Wiley series in probability and mathematical statistics, J. Wiley & Sons, 1993. ,
A Tutorial on the Cross-Entropy Method, Annals of Operations Research, vol.16, issue.3, pp.19-67, 2005. ,
DOI : 10.1137/S0363012901385691
Additive Covariance kernels for high-dimensional Gaussian Process modeling, Annales de la facult?? des sciences de Toulouse Math??matiques, vol.21, issue.3, pp.481-499, 2012. ,
DOI : 10.5802/afst.1342
URL : https://hal.archives-ouvertes.fr/hal-00644934
DiceDesign and DiceEval: two R packages for design and analysis of computer experiments, Journal of Statistical Software, vol.65, issue.11, pp.1-38, 2015. ,
Fast generation of space-filling latin hypercube sample designs, 13th AIAA/ISSMO Multidisciplinary Analysis Optimization Conference, 2010. ,
Six factors which affect the condition number of matrices associated with kriging, Mathematical Geology, vol.19, issue.3, pp.669-683, 1997. ,
DOI : 10.1007/BF02769650
Additive Gaussian processes, Advances in Neural Information Processing Systems 24, pp.226-234, 2011. ,
Using Gaussian processes to optimize expensive functions Advances in Artificial Intelligence, 21st Australasian Joint Conference on Artificial Intelligence Proceedings, pp.258-267, 2008. ,
Global optimization of deceptive functions with sparse sampling, 12th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, 2008. ,
Recent advances in surrogatebased optimization, Progress in Aerospace Sciences, pp.50-79, 2009. ,
Engineering design via surrogate modelling: a practical guide, 2008. ,
DOI : 10.1002/9780470770801
Classes of kernels for machine learning: A statistics perspective, Journal of Machine Learning Research, vol.2, pp.299-312, 2002. ,
Ensemble of surrogates. Structural and Multidisciplinary Optimization, pp.199-216, 2007. ,
DOI : 10.1007/s00158-006-0051-9
Bayesian Gaussian Processes for Regression and Classification, 1997. ,
Adaptive design and analysis of supercomputer experiments, Technometrics, vol.51, issue.2, pp.130-144, 2009. ,
Cases for the nugget in modeling computer experiments, Statistics and Computing, vol.22, issue.3, pp.713-722, 2012. ,
Computational Intelligence in Expensive Optimization Problems, chapter Kriging Is Well- Suited to Parallelize Optimization, pp.131-162, 2010. ,
Real-parameter black-box optimization benchmarking 2009: Experimental setup, 2009. ,
URL : https://hal.archives-ouvertes.fr/inria-00362649
On the efficiency of certain quasi-random sequences of points in evaluating multi-dimensional integrals, Numerische Mathematik, vol.38, issue.1, pp.84-90, 1960. ,
DOI : 10.1007/BF01386213
Generalized least squares inference in panel and multilevel models with serial correlation and fixed effects, Journal of Econometrics, vol.140, issue.2, pp.670-694, 2007. ,
DOI : 10.1016/j.jeconom.2006.07.011
Benchmarking a BI-population CMA-ES on the BBOB-2009 function testbed, Proceedings of the 11th annual conference companion on Genetic and evolutionary computation conference, GECCO '09, pp.2389-2395, 2009. ,
DOI : 10.1145/1570256.1570333
URL : https://hal.archives-ouvertes.fr/inria-00382093
The CMA Evolution Strategy: A Tutorial, 2009. ,
DOI : 10.1007/11007937_4
URL : https://hal.archives-ouvertes.fr/hal-01297037
Comparing results of 31 algorithms from the black-box optimization benchmarking BBOB-2009, Proceedings of the 12th annual conference comp on Genetic and evolutionary computation, GECCO '10, pp.1689-1696, 2010. ,
DOI : 10.1145/1830761.1830790
URL : https://hal.archives-ouvertes.fr/hal-00545727
Real- Parameter Black-Box Optimization Benchmarking 2009: Noiseless Functions Definitions, 2009. ,
URL : https://hal.archives-ouvertes.fr/inria-00362633
An evaluation of sequential model-based optimization for expensive blackbox functions, Proceeding of the fifteenth annual conference companion on Genetic and evolutionary computation conference companion, GECCO '13 Companion, pp.1209-1216, 2013. ,
DOI : 10.1145/2464576.2501592
Evaluating the CMA Evolution Strategy on Multimodal Test Functions, Parallel Problem Solving from Nature -PPSN VIII, pp.282-291 ,
DOI : 10.1007/978-3-540-30217-9_29
Adapting arbitrary normal mutation distributions in evolution strategies: the covariance matrix adaptation, Proceedings of IEEE International Conference on Evolutionary Computation, pp.312-317, 1996. ,
DOI : 10.1109/ICEC.1996.542381
Completely Derandomized Self-Adaptation in Evolution Strategies, Evolutionary Computation, vol.9, issue.2, pp.159-195, 2001. ,
DOI : 10.1016/0004-3702(95)00124-7
URL : http://www.mitpressjournals.org/userimages/ContentEditor/1164817256746/lib_rec_form.pdf
On the adaptation of arbitrary normal mutation distributions in evolution strategies: The generating set adaptation, Sixth International Conference on Genetic Algorithms, pp.312-317, 1995. ,
Adaptation in Natural and Artificial Systems, 1975. ,
Stochastic Local Search, 7th International Conference Progressive Reinforcement-Learning-Based Surrogate Selection, pp.110-124, 2004. ,
DOI : 10.1201/9781420010749.ch19
KRIging based regression and optimization Scilab Package. https://atoms.scilab.org/toolboxes, 2013. ,
Surrogate-assisted evolutionary computation: Recent advances and future challenges, Swarm and Evolutionary Computation, vol.1, issue.2, pp.61-70, 2011. ,
DOI : 10.1016/j.swevo.2011.05.001
A taxonomy of global optimization methods based on response surfaces, Journal of Global Optimization, vol.21, pp.345-383, 2001. ,
Lipschitzian optimization without the Lipschitz constant, Journal of Optimization Theory and Applications, vol.20, issue.1, pp.157-181, 1993. ,
DOI : 10.1007/BF00941892
Simultaneous kriging-based estimation and optimization of mean response, Journal of Global Optimization, vol.10, issue.4, pp.313-336, 2013. ,
DOI : 10.1007/s10898-008-9354-2
URL : https://hal.archives-ouvertes.fr/emse-00674460
Genetic and Evolutionary Computation ? GECCO 2004: Genetic and Evolutionary Computation Conference, Proceedings, Part I, chapter Reducing Fitness Evaluations Using Clustering Techniques and Neural Network Ensembles, pp.688-699, 2004. ,
Efficient global optimization of expensive black-box functions, Journal of Global Optimization, vol.13, issue.4, pp.455-492, 1998. ,
DOI : 10.1023/A:1008306431147
A robust optimization approach using kriging metamodels for BIBLIOGRAPHY robustness approximation in the CMA-ES, IEEE Congress on Evolutionary Computation, pp.1-8, 2010. ,
Optimization by Simulated Annealing, Science, vol.220, issue.4598, pp.671-680, 1983. ,
DOI : 10.1126/science.220.4598.671
URL : http://www.cs.virginia.edu/cs432/documents/sa-1983.pdf
Local Meta-models for Optimization Using Evolution Strategies, Parallel Problem Solving from Nature -PPSN IX, pp.939-948, 2006. ,
DOI : 10.1007/11844297_95
URL : http://www.bionik.tu-berlin.de/user/niko/ppsn06model.pdf
Simulation-optimization via kriging and bootstrapping: a survey, J. Simulation, vol.8, issue.4, pp.241-250, 2014. ,
A Statistical Approach to Some Basic Mine Valuation Problems on the Witwatersrand, OR, vol.4, issue.1, 1953. ,
When are Gauss-Markov and least squares estimators identical? a coordinate-free approach. The Annals of Mathematical Statistics, pp.70-75, 1968. ,
DOI : 10.1214/aoms/1177698505
URL : http://doi.org/10.1214/aoms/1177698505
An experimental methodology for response surface optimization methods, Journal of Global Optimization, vol.4, issue.1, pp.699-736, 2012. ,
DOI : 10.1017/CBO9780511809682
An evolution strategy assisted by an ensemble of local Gaussian process models, Proceeding of the fifteenth annual conference on Genetic and evolutionary computation conference, GECCO '13, pp.447-454, 2013. ,
DOI : 10.1145/2463372.2463425
Globalized Nelder-Mead method for engineering optimization, Computers & Structures, vol.82, pp.23-262251, 2004. ,
Bayesian algorithms for one-dimensional global optimization, Journal of Global Optimization, vol.10, issue.1, pp.57-76, 1997. ,
DOI : 10.1023/A:1008294716304
Surrogate-Assisted Evolutionary Algorithms. Theses, Université Paris Sud -Paris XI ; Institut national de recherche en informatique et en automatique -INRIA, 2013. ,
URL : https://hal.archives-ouvertes.fr/tel-00823882
Analysis of Computer Experiments Using Penalized Likelihood in Gaussian Kriging Models, Technometrics, vol.47, issue.2, 2005. ,
DOI : 10.1198/004017004000000671
Self-adaptive surrogate-assisted covariance matrix adaptation evolution strategy, Proceedings of the fourteenth international conference on Genetic and evolutionary computation conference, GECCO '12, pp.321-328 ,
DOI : 10.1145/2330163.2330210
URL : https://hal.archives-ouvertes.fr/hal-00686570
Intensive surrogate model exploitation in self-adaptive surrogate-assisted cma-es (saacm-es), Proceeding of the fifteenth annual conference on Genetic and evolutionary computation conference, GECCO '13, pp.439-446, 2013. ,
DOI : 10.1145/2463372.2463427
URL : https://hal.archives-ouvertes.fr/hal-00818595
A Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output From a Computer Code, Technometrics, vol.15, issue.1, pp.55-61, 2000. ,
DOI : 10.1214/aoms/1177703566
An analytic comparison of regularization methods for Gaussian Processes, 2016. ,
A detailed analysis of kernel parameters in Gaussian process-based optimization, Ecole Nationale Supérieure des Mines, 2015. ,
URL : https://hal.archives-ouvertes.fr/hal-01246677
Design and analysis of experiments, 2001. ,
The application of bayesian methods for seeking the extremum, Towards Global Optimization, vol.2, issue.2, pp.117-129, 1978. ,
Monte Carlo Implementation of Gaussian Process Models for Bayesian Regression and Classification, 1997. ,
A simplex method for function minimization, Computer Journal, vol.7, pp.308-313, 1965. ,
Gaussian processes for global optimization, Proceedings of the 3rd Learning and Intelligent OptimizatioN Conference, 2009. ,
Probabilistic sensitivity analysis of complex models: a Bayesian approach, Journal of the Royal Statistical Society: Series B (Statistical Methodology), vol.34, issue.3, pp.751-769, 2002. ,
DOI : 10.1214/ss/1009213004
Curse and blessing of uncertainty in evolutionary algorithm using approximation, Evolutionary Computation, pp.2928-2935, 2006. ,
The role of the nugget term in the Gaussian process method In mODa 9 ? Advances in Model-Oriented Design and Analysis, pp.149-156, 2010. ,
Restarted Local Search Algorithms for Continuous Black Box Optimization, Evolutionary Computation, vol.15, issue.11, pp.575-607, 2012. ,
DOI : 10.1016/0004-3702(95)00124-7
UOBYQA: unconstrained optimization by quadratic approximation, Mathematical Programming, pp.555-582, 2002. ,
DOI : 10.1007/s101070100290
URL : http://www.damtp.cam.ac.uk/user/na/NA_papers/NA2000_14.ps.gz
The BOBYQA algorithm for bound constrained optimization without derivatives, 2009. ,
Recovery of inter-block information when block sizes are unequal, Biometrika, vol.58, issue.3, p.545, 1971. ,
DOI : 10.1093/biomet/58.3.545
A benchmark of kriging-based infill criteria for noisy optimization. Structural and Multidisciplinary Optimization, pp.607-626, 2013. ,
URL : https://hal.archives-ouvertes.fr/hal-00658212
Surrogate-based analysis and optimization, Progress in Aerospace Sciences, pp.1-28, 2005. ,
Subset selection from large datasets for kriging modeling. Structural and Multidisciplinary Optimization, pp.545-569, 2009. ,
DOI : 10.2139/ssrn.1104595
DiceKriging, DiceOptim: Two R packages for the analysis of computer experiments by krigingbased metamodeling and optimization, Journal of Statistical Software, vol.51, issue.1, pp.1-55, 2012. ,
URL : https://hal.archives-ouvertes.fr/hal-00495766
A Computationally Stable Approach to Gaussian Process Interpolation of Deterministic Computer Simulation Data, Technometrics, vol.53, issue.4, pp.366-378, 2011. ,
DOI : 10.1198/TECH.2011.09141
RISE: An Incremental Trust-Region Method for Robust Online Sparse Least-Squares Estimation, IEEE Transactions on Robotics, vol.30, issue.5, pp.1091-1108, 2014. ,
DOI : 10.1109/TRO.2014.2321852
URL : http://people.csail.mit.edu/kaess/pub/Rosen14tro.pdf
Applied Regression Analysis: A Research Tool, 1998. ,
DOI : 10.1007/b98890
Derivative-free optimization: a review of algorithms and comparison of software implementations, Journal of Global Optimization, vol.18, issue.3, pp.1247-1293, 2013. ,
DOI : 10.1088/0953-8984/18/39/002
Principles of mathematical analysis, International Series in Pure and Applied Mathematics, 1976. ,
On correlated mutations in evolution strategies, Proceedings of the 2nd Conference on Parallel Problem Solving from Nature, pp.107-116, 1992. ,
Gaussian Processes for Machine Learning Adaptive Computation and Machine Learning, 2005. ,
Flexibility and Efficiency Enhancements For Constrainted Global Design Optimization with Kriging Approximations, 2002. ,
Factorial hypercube designs for spatial correlation regression, Journal of Applied Statistics, vol.24, issue.4, pp.453-474, 1997. ,
DOI : 10.1080/02664769723648
Computer experiments and global optimization, 1997. ,
Extending population-based incremental learning to continuous search spaces, Proceedings of the 5th International Conference on Parallel Problem Solving from Nature, PPSN V, pp.418-427, 1998. ,
DOI : 10.1007/BFb0056884
URL : https://hal.archives-ouvertes.fr/hal-00116542
A sequential method seeking the global maximum of a function, SIAM Journal on Numerical Analysis, vol.9, issue.3, pp.379-388, 1972. ,
A parallel updating scheme for approximating and optimizing high fidelity computer simulations. Structural and Multidisciplinary Optimization, pp.371-383, 2004. ,
On the Design of Optimization Strategies Based on Global Response Surface Approximation Models, Journal of Global Optimization, vol.27, issue.5, pp.31-59, 2005. ,
DOI : 10.1007/978-1-4613-1997-9
On the distribution of points in a cube and the approximate evaluation of integrals, USSR Computational Mathematics and Mathematical Physics, vol.7, issue.4, pp.86-112, 1967. ,
DOI : 10.1016/0041-5553(67)90144-9
A tutorial on support vector regression, Statistics and Computing, vol.14, issue.3, pp.199-222, 2004. ,
DOI : 10.1023/B:STCO.0000035301.49549.88
Linear Algebra and Its Applications, 1988. ,
Introduction to Linear Algebra, 2009. ,
Tros- set. MAPS: Model-assisted pattern search, 1997. ,
Design and Analysis of Computer Experiments, Statistical Science, vol.4, issue.4, pp.433-435, 1989. ,
DOI : 10.1214/ss/1177012413
The Design and Analysis of Computer Experiments. Springer series in statistics, 2003. ,
A Bayesian approach to sequential optimization based on computer experiments ,
Evolution strategies assisted by Gaussian processes with improved pre-selection criterion, Proceedings of the 2003 Congress on Evolutionary Computation CEC2003, pp.692-699, 2003. ,
DOI : 10.1109/cec.2003.1299643
URL : http://www-ra.informatik.uni-tuebingen.de/publikationen/2003/ulmer03evolution.pdf
Efficient global optimization algorithm assisted by multiple surrogate techniques, Journal of Global Optimization, vol.56, issue.2, pp.669-689, 2013. ,
An informational approach to the global optimization of expensive-to-evaluate functions, Journal of Global Optimization, vol.10, issue.5, pp.509-534, 2008. ,
DOI : 10.1007/978-1-4612-1494-6
URL : https://hal.archives-ouvertes.fr/hal-00354262
An informational approach to the global optimization of expensive-to-evaluate functions, Journal of Global Optimization, vol.10, issue.5, pp.509-534, 2009. ,
DOI : 10.1007/978-1-4612-1494-6
URL : https://hal.archives-ouvertes.fr/hal-00354262
A subjective review of the state of the art in model-based parameter tuning, the Workshop on Experimental Methods for the Assessment of Computational Systems (WEMACS), 2010. ,
Inverting Modified Matrices. Number 42 in Statistical Research Group Memorandum Reports, 1950. ,
Bayesian optimization in high dimensions via random embeddings, International Joint Conferences on Artificial Intelligence (IJCAI), 2013. ,
Asymptotic properties of a maximum likelihood estimator with data from a Gaussian process, Journal of Multivariate Analysis, vol.36, issue.2, pp.280-296, 1991. ,
DOI : 10.1016/0047-259X(91)90062-7
Kriging metamodel with modified nuggeteffect: The heteroscedastic variance case, Computers & Industrial Engineering, issue.3, pp.61760-777, 2011. ,
DOI : 10.1016/j.cie.2011.05.008
Kriging and cross-validation for massive spatial data, Environmetrics, vol.23, issue.1, pp.290-304, 2010. ,
DOI : 10.1007/978-1-4612-1494-6