Theory of Iterative Optimization Heuristics: From Black-Box Complexity over Algorithm Design to Parameter Control
Résumé
Sampling-based optimization heuristics are algorithms which aim to find good solutions for problems that cannot be solved through exact solution strategies. They are particularly useful for the optimization of problems that are not given as explicit formulae, but which require computer simulations or physical experiments to assess the quality of a proposed solution. In such settings, sampling-based optimization heuristics are essentially the only means to obtain good solutions. Another important application of sampling-based optimization heuristics is in the optimization of problems that are too complex to be solved analytically, and which remain intractable for problem-specific algorithm designs.
An important sub-class of sampling-based optimization heuristics are iterative optimization heuristics (IOHs). IOHs aim at finding good solutions by a sequential process of evaluating solution candidates, using the obtained information to adjust the strategy by which the next samples are generated, and iterating this process until a termination criterion is met.
With my research, I aim to contribute to our understanding of the working principles that drive the performance of IOHs. More precisely, I aim at understanding which strategies work well for which types of problems. To this end, I study the efficiency of existing IOHs, and I compare these performances to that of a best possible solver for the given problem. For the latter, I develop and analyze complexity models that allow us to quantify the influence of certain algorithmic choices, such as the size of memory used, the strategy by which solutions can be sampled, the type of information used to select the next sampling strategies, etc.
This thesis summarizes some of my research results obtained in the last nine years, since the completion of my PhD studies. We first derive improved (and often tight) bounds for the black-box complexity of the two best known benchmark problems in the theory of IOHs, OneMax and LeadingOnes, and this for various black-box models. We then demonstrate how insights obtained from such black-box complexity studies can inspire the design of efficient optimization techniques. An important result obtained through this approach is a rigorous example for a situation in which a non-static choice of the control parameters of an IOH yields a super-constant performance gain over any possible static parameter setting. This result has revived theoretical research on parameter control, which has seen significant advances since. We discuss some examples in this thesis. We conclude by discussing the role of algorithm benchmarking as a bridge between theoretical and empirical research of IOHs.
Origine : Fichiers produits par l'(les) auteur(s)