Agile, baby!

One of the purposes of this blog was the battle of better software quality in our everyday lives.

Polish government websites in the past tend to be famous for their lack of testing and missing the standard quality levels, but it was in the past. After several years form debut such amazing pieces of software as PKP or ZUS sites, I naively thought that the situation got better.

Ha!

Alarmed by the article stating that every taxpayer in Poland should generate their own bank number to be able to pay taxes next year , I decided to go to the government website https://www.podatki.gov.pl and generate it for myself.

Such a brilliant occasion for testing! #NeverStopTesting

The idea of the bank account number generation is that you enter your PESEL number (the unique number, that every polish citizen has).

The PESEL is an 11-digit number, containing one’s date of birth.

What you are supposed to do in order to follow the instructions:

  1. Open the website https://www.podatki.gov.pl/generator-mikrorachunku-podatkowego/
  2. Select PESEL
  3. Type your PESEL number

Expected result: Account number is generated.

nice and smooth. Worked for me. 🙂

https://i.pinimg.com/736x/13/ff/49/13ff49773ca9c25ac2116c8bc6c4d2ee.jpg

…. and what you are supposed to do when you are a tester 😀

“A Software Tester walks into a bar….”

One of my testing scenarios:

  1. Enter curl …
    for example: curl https://www.podatki.gov.pl/umbraco/surface/TaxAccountNumberSearch/GetAccountNumberByPesel?pesel=9223372036854775807
  2. Click Enter

Expected result:

What does it mean? The UI of this website, as i tested, was protected from entering -, a, 8946180480104610401 or anything else into input field. good job! On the other hand, the Backed…..

Well, above error speaks for itself. But I decided to explain it in a bit more detailed way:

  • There is no data validation on the backed side
  • Unnecessary information displayed in public
    => in the error’s stack trace
  • The code has been compiled in the debug version (including all information about the source code – you can see it in the file names and rows in the stack trace) deployed in production
    => unnecessary leak of information plus worse performance of the site. The code in debug version is usually not optimized and has additional instructions
  • It is likely that the code has been compiled of the developer’s machine and deployed in production (Mr. Jedrzej Lenart?)
    => it is a violation of security rules – you have no guarantee that the local changes has not been pushed into production as well
  • The website calls this web-service by POST (which is correct), but in the browser and in curl you can see that the parameter “pesel" is called by GET (in the URL)
    => it is not the worst solution, but it usually makes the website more prone to CSFR attacks and the PESEL number is visible in: logs of servers and routers – even when the communication is encrypted (URLs are not encrypted).

Why

It seems that someone in the ministry of Finance IT department deploys code without testing. Bravo for Agile!

Bravo for Continuous Delivery!

Oh why

On the other hand, someone deploys code on GOVERMENT website without testing. Code prone for attack.

It is possible that they were not aware of the micro-service working below, but still…

Funny fact

When I posted that on Twitter it took couple of minutes in the Ministry of Finance for displaying good 500:

So Agile again!

Just wanted to share it with you, because Mr. Jędrzej made my day. (info in the screenshot).

In case of any comments stalk me on Twitter. Cheers!

Why do you need mutation testing?

I bet that you wonder every day how to improve your testing and found issues before the software gets into production.

One of the great ideas of trying something new is to transform your tests scripts into real mutants 🙂

What is mutation testing?

Mutation testing is conceptually quite simple.

Faults (or mutations) are automatically seeded into the code, then the tests are run. If tests fail, then the mutation is killed, but when the tests pass then the mutation survives. 😉

The quality of the automation test scripts can be gauged from the percentage of killed mutations.|Fun!

A mutator is a change in the program logic, e.g. when it comes to conditionals: The “Negate Conditionals Mutator” changes conditional expressions into their opposite. An expression like

if (a == b) {
  // do something

}

will be mutated to
if (a != b) {
  // do something

}

https://vignette.wikia.nocookie.net/despicableme/images/5/52/Screenshot_2016-02-10-01-09-16.jpg/revision/latest?cb=20161028002525

What?

In one of my projects we use PiTest (or PIT) as a tool for mutation testing.
It is an open-source framework for mutation testing in Java, so everyone can try it out.

PIT runs your unit tests against automatically modified versions of the application code. When the application code changes, it should produce different results and cause the unit tests to fail. On the other hand, if a unit test does not fail in this situation, it may indicate an issue with the test suite.

Why?

Traditional test coverage (i.e line, statement, branch, etc.) measures only which code is executed by your tests. It does not check if your tests are actually able to detect faults in the executed code. It is therefore only able to identify code that is definitely not tested.

The most extreme examples of the problem are tests with no assertions. Fortunately, these are uncommon in most code bases. Much more common is code that is only partially tested by its suite. A suite that only partially tests code can still execute all its branches (examples).

As it is actually able to detect whether each statement is meaningfully tested, mutation testing is the gold standard against which all other types of coverage are measured.

For sure, PiTest is not the only tool for mutation testing, but it is fast and reliable, so I can recommend it for your project. What is more, it is open-source, but it is actively developed and supported, so it is quite likely that you will not be left alone with your problems with the tool.

PiTest works well with development toolings such as Maven or IntelliJ.
The reports produced by PIT are in an easy to read format combining line coverage and mutation coverage information. In addition, it provides valuable insights into the quality of tests and helps to efficiently identify test gaps which need to be closed.

Is it for everybody?

Use of unit tests is part of common development approach. Teams are measuring code coverage (condition and branch) usually as a part of the Definition of Done. On the other hand, this measurement doesn’t check the quality of the tests. In extreme situations it may happen that we just increase coverage without testing something meaningful.

This is where mutation testing kicks in.

The basic idea is that a good test should not work anymore when the underlying code is changed, “mutated”. 

https://i.ytimg.com/vi/Z0-4tA6UCVE/maxresdefault.jpg

PITEST performs for each class in scope suitable mutations on bytecode-level and checks if the associated unit tests fail or not. It then reports out the cases where mutations survived the test, either due to quality of the test or due to a missing test, and the cases where mutations made the tests fail. The result is a coverage report (one per module), which summarises mutation coverage and allows to drill-down.

How to integrate PITest

Maven

PITest can be added as a plug-in to Maven

<plugin>     <groupId>org.pitest</groupId>     <artifactId>pitest-maven</artifactId>     <version>LATEST</version> </plugin>

The Maven command line is

mvn org.pitest:pitest-maven:mutationCoverage

More details on Maven configuration options can be found here: http://pitest.org/quickstart/maven/

With the above basic configuration, html-reports of the form shown above are created for each Maven module.

In the context of Halloween, I hope you’ll give a chance to the mutants and try to improve your testing.

In case of any comments stalk me on Twitter or comment down below. Cheers!

A screenshot of a social media post

Description automatically generated