University of Twente Student Theses

Login

Faster Mutation Testing through Simultaneous Mutation Testing

Roos, M.C. de (2024) Faster Mutation Testing through Simultaneous Mutation Testing.

This is the latest version of this item.

[img] PDF
1MB
Abstract:Mutation testing is a powerful fault-based testing technique, which can be used to evaluate the quality of a test suite for a specific program. This is done by injecting faults into the program, called mutations. Programs injected with mutations are called mutants and will typically behave differently from the original program, and hence (given that the original program supposedly implements the desired functionality) are faulty by construction. The idea is that an adequate test suite of high quality would be able to detect these mutants by means of a failing test, whereas a low quality test suite would not. Unfortunately, mutation testing is costly in terms of time as it needs to run the test suite for each generated mutant. Although many optimisations already exist, most of them come with a trade-off between improving performance and reducing quality. This project attempts to improve performance while retaining quality, by testing mutants as a group, as opposed to individually, which is called simultaneous mutation testing. For this purpose, a theoretical view on the cost of mutation testing is presented alongside the theoretical performance benefit of using simultaneous mutation testing over regular mutation testing. Additionally, two algorithms for grouping mutants were explored, one of which is rather pragmatic while the other uses a constraint solver. We implemented simultaneous mutation testing in StrykerJS. To validate the theory and explore the difference in performance between various groups, an experiment was set up. The experiment shows that simultaneous mutation testing can improve the performance of the mutation testing tool StrykerJS by 3%. This minor improvement can be explained by the fact that the time spent creating test sessions in StrykerJS is relatively small. The follow-up experiment, in which we increase the time spent creating test sessions, shows a performance improvement of 27%. Simultaneous mutation testing retains over 99.9% of the quality of mutation testing. Overall, the pragmatic algorithm performs better than the solver algorithm.
Item Type:Essay (Master)
Clients:
Info Support, Veenendaal, Netherlands
Faculty:EEMCS: Electrical Engineering, Mathematics and Computer Science
Subject:54 computer science
Programme:Computer Science MSc (60300)
Link to this item:https://purl.utwente.nl/essays/98905
Export this item as:BibTeX
EndNote
HTML Citation
Reference Manager

 

Repository Staff Only: item control page