Why do you need mutation testing?

I bet that you wonder every day how to improve your testing and found issues before the software gets into production.

One of the great ideas of trying something new is to transform your tests scripts into real mutants 🙂

What is mutation testing?

Mutation testing is conceptually quite simple.

Faults (or mutations) are automatically seeded into the code, then the tests are run. If tests fail, then the mutation is killed, but when the tests pass then the mutation survives. 😉

The quality of the automation test scripts can be gauged from the percentage of killed mutations.|Fun!

A mutator is a change in the program logic, e.g. when it comes to conditionals: The “Negate Conditionals Mutator” changes conditional expressions into their opposite. An expression like

if (a == b) {
  // do something

}

will be mutated to
if (a != b) {
  // do something

}

https://vignette.wikia.nocookie.net/despicableme/images/5/52/Screenshot_2016-02-10-01-09-16.jpg/revision/latest?cb=20161028002525

What?

In one of my projects we use PiTest (or PIT) as a tool for mutation testing.
It is an open-source framework for mutation testing in Java, so everyone can try it out.

PIT runs your unit tests against automatically modified versions of the application code. When the application code changes, it should produce different results and cause the unit tests to fail. On the other hand, if a unit test does not fail in this situation, it may indicate an issue with the test suite.

Why?

Traditional test coverage (i.e line, statement, branch, etc.) measures only which code is executed by your tests. It does not check if your tests are actually able to detect faults in the executed code. It is therefore only able to identify code that is definitely not tested.

The most extreme examples of the problem are tests with no assertions. Fortunately, these are uncommon in most code bases. Much more common is code that is only partially tested by its suite. A suite that only partially tests code can still execute all its branches (examples).

As it is actually able to detect whether each statement is meaningfully tested, mutation testing is the gold standard against which all other types of coverage are measured.

For sure, PiTest is not the only tool for mutation testing, but it is fast and reliable, so I can recommend it for your project. What is more, it is open-source, but it is actively developed and supported, so it is quite likely that you will not be left alone with your problems with the tool.

PiTest works well with development toolings such as Maven or IntelliJ.
The reports produced by PIT are in an easy to read format combining line coverage and mutation coverage information. In addition, it provides valuable insights into the quality of tests and helps to efficiently identify test gaps which need to be closed.

Is it for everybody?

Use of unit tests is part of common development approach. Teams are measuring code coverage (condition and branch) usually as a part of the Definition of Done. On the other hand, this measurement doesn’t check the quality of the tests. In extreme situations it may happen that we just increase coverage without testing something meaningful.

This is where mutation testing kicks in.

The basic idea is that a good test should not work anymore when the underlying code is changed, “mutated”. 

https://i.ytimg.com/vi/Z0-4tA6UCVE/maxresdefault.jpg

PITEST performs for each class in scope suitable mutations on bytecode-level and checks if the associated unit tests fail or not. It then reports out the cases where mutations survived the test, either due to quality of the test or due to a missing test, and the cases where mutations made the tests fail. The result is a coverage report (one per module), which summarises mutation coverage and allows to drill-down.

How to integrate PITest

Maven

PITest can be added as a plug-in to Maven

<plugin>     <groupId>org.pitest</groupId>     <artifactId>pitest-maven</artifactId>     <version>LATEST</version> </plugin>

The Maven command line is

mvn org.pitest:pitest-maven:mutationCoverage

More details on Maven configuration options can be found here: http://pitest.org/quickstart/maven/

With the above basic configuration, html-reports of the form shown above are created for each Maven module.

In the context of Halloween, I hope you’ll give a chance to the mutants and try to improve your testing.

In case of any comments stalk me on Twitter or comment down below. Cheers!

A screenshot of a social media post

Description automatically generated

Your Internet browsing leads to greenhouse gases (GHG) emissions

What’s the occasion for such geeky title?

Tester’s Day and Programmers Day! Best wishes to you all.

Using the celebration time, I would like to pint out the matters that bother me for some time: greenhouse gas emission, electricity consumption, ecology and the care for our planet in general.

Do you remember this old-fashioned message at the bottom of each email that we started to put in the email footer to prevent it from ruining forests? It was the time when people discovered that there is no point in using printed messages, when we can exchange them virtually.

This step pro-nature was great and changed people’s way of thinking.

A letter was no longer a hand-written piece of paper, that requires a post to be delivered, it started to be easy, quick and accessible in real-time for all internet users.

On the other hand, with the growing networks of friends, or employees in the companies we work in, the need for easy, but sometimes pointless communication, flourished and become our daily routine.

When a vintage letter was sent over to a person, it usually consisted of important news, documents or an ask for some action. Today, on our private accounts, we receive multiple email messages advertising something, informing about events we don’t need or form people we don’t know. At work, it is even worst. Sometimes I have the impression that HR departments, in their goodwill, sent multiple emails to LITERALLY everybody in the company’s mailboxes due to information about some events or actions we don’t need or don’t care about.

It wouldn’t be so bad, if it was just a matter of bothering people – we all have a SPAM folder for it.

Today though, I would like you to take a look at the completely different angle of sending out tons of spam. Electricity.

80% of the energy consumed in the world comes from fossil fuels, and CO2 accounts for the great majority of our greenhouse gas emissions. Electricity is not a magic power that runs computer – it has it’s the very concrete source in the ground – no matter if we use coal, gas or any other fuel to produce it. Where is the connection between sending emails TO ALL and electricity? Think twice.

Pointless emails are not the only sources of wasting electricity. there are web resources, such as videos, pictures, hi-tech content, that use more energy than any old website would. The more fireworks you have on your website – the more energy it consumes. The more people watch it – the more greenhouse gasses fly to the atmosphere. It is really sad, because each of us would like to be modern and hi-tech, would like to have the best websites they can and be the most influential. What we need to remember is that the price for it is high.

The same is with our modern needs – constant browsing. The more you browse – the more energy it takes. Not only at the tour end, but definitely at the end of all those poor servers, which host your favorite websites and services. I know that this is how the modern world works, but I also think that this is something that we don’t think much about. When we scroll Facebook feed – we sometimes think that it consumes (wastes) our time – not necessarily that it consumes piles of bytes and energy.

To learn more about climate change and IT impact, I encourage you to check your website’s impact on the environment, using Firefox addon – Carbonalyser. I also encourage you to write fewer emails and just in really important matters. just be kind to our Planet.

In case of any comments – go to the comment section below or stalk me on Twitter, and yes, I’ve removed my Facebook account. This is my little step.

Cheers!