Investigate the matrix : leveraging variability to specialize software and test suites

Paul Temple 1
1 DiverSe - Diversity-centric Software Engineering
Inria Rennes – Bretagne Atlantique , IRISA-D4 - LANGAGE ET GÉNIE LOGICIEL
Abstract : Nowadays, software have to be efficient, fast to execute, etc. They can be configured to adapt to specific needs. Each configuration leads to a different system and usually it is hard to generate them all. Thus, the exhaustive evaluation of their performance is impossible. Furthermore, several executions of systems, under different conditions, are needed to properly evaluate performances. Two dimensions emerge from this description of performance testing: the selection of system configurations allowing to generate associated systems that meet expressed needs and the selection of test cases allowing to observe performances of systems under different conditions. We propose to represent those two dimensions as a (performance) matrix: one dimension represents selected systems that can be observed while the other dimension represents the set of test cases that will be executed on each of these systems. Each cell is the execution of a program variant regarding a test. The contributions of this thesis are as follows : First, we leverage Machine Learning techniques in order to specialize a Software Product Line (in this case a video generator) helping in selecting a configuration that is likely to meet requirements. End users must be able to express their requirements such that it results in a binary decision problem (i.e., configurations that are acceptable and those that are not). Machine Learning techniques are then used to retrieve partial configurations that specialize a Software Product Line to guide end users and reduce the configuration space. In the end, this work aims at diminishing the first dimension of the matrix that deals with systems and programs. Second, we propose a new method assessing the ability of test suites to reveal significant performance differences of a set of configurations tackling the same task. This method can be used to assess whether a new test case is worth adding to a test suite or to select an optimal test set with respect to a property of interest. In the end, it might help structuring the execution of tests. For instance, it can create an order of execution resulting in using less test cases that are presented in the second dimension of the matrix. We evaluated our approach on several systems from different domains such as OpenCV or Haxe.
Keywords : Tests Matrix Software
Document type :
Theses
Complete list of metadatas

Cited literature [141 references]  Display  Hide  Download

https://tel.archives-ouvertes.fr/tel-02125962
Contributor : Abes Star <>
Submitted on : Friday, May 10, 2019 - 5:14:06 PM
Last modification on : Friday, September 13, 2019 - 9:48:41 AM

File

TEMPLE_Paul.pdf
Version validated by the jury (STAR)

Identifiers

  • HAL Id : tel-02125962, version 1

Citation

Paul Temple. Investigate the matrix : leveraging variability to specialize software and test suites. Software Engineering [cs.SE]. Université Rennes 1, 2018. English. ⟨NNT : 2018REN1S087⟩. ⟨tel-02125962⟩

Share

Metrics

Record views

533

Files downloads

76