2019 AIM HDC

How good are your tests? Would they still pass if the tested code was changed? If so, there may be problems with your code, your tests, or both!  Mutation Testing helps reveal these cases. It makes lots of slightly altered versions, called “mutants”, of each of your functions or methods, and runs each method/function’s unit tests, using each of its mutants instead. If a mutant makes any test fail, that mutant “dies”. If there are any “survivors”, that implies that there are flaws in your code, your tests, or both! Your code might not be meaningful enough that a slight mutation would change the behavior, your tests might not be strict enough to catch the difference that the mutation made, or both!

This talk will introduce you to mutation testing, its benefits, drawbacks, and pitfalls, and how it works under the hood. There will be several examples (in clear pseudocode), and a list of tools for popular languages (including Python, Ruby, Javascript, Java, C++, C#, anything that compiles via LLVM, and more).  You will come away equipped with a powerful new technique for making sure your tests are strict and your code is meaningful.