Skip to Main content Skip to Navigation

Automatic Unit Test Amplification For DevOps

Benjamin Danglot 1, 2
2 SPIRALS - Self-adaptation for distributed services and large software systems
Inria Lille - Nord Europe, CRIStAL - Centre de Recherche en Informatique, Signal et Automatique de Lille (CRIStAL) - UMR 9189
Abstract : Over the last decade, strong unit testing has become an essential component of any serious software project, whether in industry or academia. The agile development movement has contributed to this cultural change with the global dissemination of test-driven development techniques. More recently, the DevOps movement has further strengthened the testing practice with an emphasis on continuous and automated testing. However, testing is tedious and costly for industry: it is hard to evaluate return on investment. Thus, developers under pressure, by lack of discipline or time might skip writing the tests. To overcome this problem, research investigates the automation of creating strong tests. The dream was that a command-line would give you a complete test suite, that verifies the whole program. Even if automatically generated test suites achieve high coverage, there are still obstacles on the adoption of such techniques by the industry. This can be explained by the difficulties to understand, integrate and maintain generated test suite. Also, most of the tools rely on weak or partial oracles, \eg absence of run-time errors, which limits their ability to find bugs. In this thesis, I aim at addressing the lack of a tool that assists developers in regression testing. To do so, I use test suite amplification. In this thesis, I define test amplification and review research works that are using test amplification. Test amplification consists of exploiting the knowledge of test methods, in which developers embed input data and expected properties, in order to enhance these manually written tests with respect to an engineering goal. In the state of the art, I reveal main challenges of test amplification and the main lacks. I propose a new approach based on both test inputs transformation and assertions generation to amplify the test suite. This algorithm is implemented in a tool called DSpot. In this thesis, I evaluate DSpot on open-source projects from GitHub. First, I improve the \ms of test suites and propose these improvements to developers through pull-requests. This evaluation shows that developers value the output of \dspot and thus accepted to integrate amplified test methods into their test suite. This proves that DSpot can improve the quality of real projects' test suites. Second, I use DSpot to detect the behavioral difference between two versions of the same program particularly to detect the behavioral change introduced by a commit. This shows that \dspot can be used in the continuous integration to achieve two crucial tasks: 1) generate amplified test methods that specify a behavioral change; 2) generate amplified test methods to improve the ability to detect potential regressions. In this thesis, I also expose three transversal contributions, related to the correctness of program. First, I study the programs' correctness under runtime perturbations. Second, I study the presence of pseudo-tested methods that are methods revealing weaknesses of the tests. Third, I study overfitting patches and test generation for automatic repair.
Document type :
Complete list of metadatas

Cited literature [136 references]  Display  Hide  Download
Contributor : Benjamin Danglot <>
Submitted on : Friday, December 6, 2019 - 10:10:05 AM
Last modification on : Friday, October 23, 2020 - 4:58:41 PM
Long-term archiving on: : Saturday, March 7, 2020 - 12:56:01 PM


Files produced by the author(s)


  • HAL Id : tel-02396530, version 1


Benjamin Danglot. Automatic Unit Test Amplification For DevOps. Software Engineering [cs.SE]. Université de Lille, 2019. English. ⟨tel-02396530⟩



Record views


Files downloads