BlackFriday

Last Friday – I was a hunter – I knew something should go wrong.
But, let’s get back to square one.

My inspiration

Tomek Dubikowski inspired me with his talk “Proper gun makes testing fun” to think more about performance testing, as lack of those would concern any of us in the most surprising situations. Following slide from his presentations would give you very simple guide of what every tester should check, to make sure, that his website or any other service will survive unexpected circumstances.

20171117_162109

This post won’t be a guideline about how to setup your environment in order to test performance of your service – but rather a tour through bad examples and good practices in performance testing.

I am not and expert – I am just gathering knowledge!

Black Friday

Black Friday is quite new term in Poland. It started to gain on popularity a year or two ago, maybe, but it became quite on spot last week. All regular shops in malls and most of online shops with clothing, cosmetics or electronics started 20% discount weekend.
I am a mean tester, so I decided to start my hunt in the evening, when traffic in online shops usually goes high. What did I hunt for? Dramatic errors, of course.

As you can guess, such discounts encouraged more customers to visit online shops and do the shopping. For those, that has not been prepared, it did not ended well. Most websites that I’ve visited were significantly slower than usual, they had some visible issues or problems with loading pictures. Unfortunately, some of them did not pass the load tests at all.

I’ll provide you with three examples of what might happen, if you won’t test performance of your websites, I hope you’ll prepare better for Christmas Sales 😀 (or whatever you wish).

All of them are taken from online stores – so the ‘only’ effects might be the shortage in store’s income and customers’ trust for the brand.

When such bloomer happens to some notional institution – it may end up in the news and in multiple memes. In 2014 we had this situation with counting votes in regional elections. Our government decided to introduce online vote transfer from local committees which ended up with tremendous fail.

As the system was not supporting high loads of transfers (you had this one job), we’ve been waiting for election’s results for a week.

One of my favorite memes from that annoying time – “It works on my machine”:

pkw

Let’s get back to Black Friday issue, shall we?

What can I do

I would divide those performance problems into two groups (both customer – facing):

  1. It doesn’t work and you have a real problem here
  2. Tell it to your customer in some human – readable way

Don’t get me wrong – if that happens to the service of yours – you’re f*** anyway 😀 It depends on you, though, how do you communicate it to your customer and if he would eventually come back to you or not.

If your service doesn’t have performance checked – too many visitors result with too many transactions for it’s capacity, which finally terminates the service with timeout. Reserved.com ‘died ‘without a warning and prompted us with extremely professional ‘504 Gateway Time-out‘.

reserved

Douglas.pl died as well, but somebody had at least done his job on UI beforehand and displayed a user-friendly information ‘We are sorry’. Better – but still…

douglas2

Most of their sites displayed this kind of messages anyway. At least, they tried.

douglas

One of our Polish Facebook Group about testing –  “Testowanie Oprogramowania” member – Artek Zwoliński – spotted fantastic fail as well at X-KOM – online store. X-kom informed us even what kind of database do they use as well 🙂

23926328_2024222767860377_8091906648219521639_o

We may make fun of it, but those examples show real problem with lack of performance testing and lack of effective solutions for peaks and unusual loads on websites. On the one hand, brand owners may claim that those are unique situations within the year and performance testing is too expensive to do it, on the other hand it is just risky for the brand and customer’s trust.

But how?

Performance testing becomes quite trendy thing now, so you can easily use online courses or watch some tutorials about how to get along with JMeter or Gatling.  However, these are just the tools. It is you – who should come up with ideas and foresee the need of testing performance. Do the risk analysis with your business and try to convince them about the need and benefits.

In addition, do your homework and don’t be sloppy in preparations. Equip yourself with knowledge and prepare (buy) an isolated environment, that will be 100% accurate to your production. If not – performance testing does not have sense, sorry. No matter how much will you invest in learning JMeter’s functions (it is already old-fashioned tool BTW).

Isolated environment is the key to successful performance testing, otherwise, happy with meaningless results, you’ll be testing your service live with end users, as many online shops did on Black Friday. 🙂

We, as testers, must be aware of such risks and know what might happen if we skip performance testing part, especially, when we develop a service, which has even a slight chance to be on top for some reason (sale, promotion, unique product etc).

Last, but not least, preserve any results of performance testing, as it may be useful in the future. The best idea is to store all the results directly in the database (for example in MongoDB) in order to have it accessible and ready for analysis.

Cheat sheet of performance testing

  1. Know your PURPOSE
  2. Isolate the environment
  3. Environment must be 100% same with production
  4. Optimise the configuration
  5. Store data
  6. Repeat the test at least once
  7. Get professionals to analyze your results

Yeah, I should get some shiny table for that 🙂

I hope it was useful and will guide you a little more into performance testing.
In case of any comments – let me know down below or stalk me on Twitter of Facebook.

Cheers!

Advertisements

Black Friday – the best performance live testing ever

Posted in development, mobile testing, production bug

Development in rapid-delivery era.

This post will consist of some thoughts that bother me for some time now. It won’t be about testing only, but about the whole modern process of software delivery. Reading shall take some time, so grab some sandwiches, coffee and come along with my story 🙂

Development in rapid - delivery era

We produce more and more software

Several years ago, it started to be visible that we live faster. We eat in a hurry, don’t care about cooking or running out of global resources. We became dedicated to meaningless things, such as smartphones, watches and gadgets to impress our society.

On the other hand, at some point, people noticed that the quality and durability of equipment they have at their homes: cars, washing machines, mobile phones etc. is lower than in used to be. It became more efficient to buy a new thing than to repair the old one. We started to produce lot of waste, polluting the environment and thinking that we have another Earth to move to. It took some time to notice, that, in fact, we don’t have a spare one. Simple, but surprising thought, made us be more eco-minded and aware of human’s impact on the planet.

Slow lifestyle, vegan food, bikes instead of cars, repairing old furniture, producing new items in more environmental friendly way – it is all happening now, and we keep on changing our lifestyles in order to save the planet for the future generations.

What has software industry to do with it?

The more devices we produce – the more software it requires. Modern fridges, microwaves or even toys tend to be equipped with some sort of software, can have WI-FI connection or are prone to be hacked. It means, that IT industry is no longer separated branch on the marked, or there that there is a bunch of guys somewhere, who produce software for your computer. Internet of Things enabled us to have software almost everywhere, furthermore, someone must write and test it.

Here is the point of my first doubt – if some device consists of some sort of software – does it mean that is has to be tested by somebody?

We test, but we don’t fix

My career as a software tester is not very long now, but since now, I had an opportunity to work is several projects related to various parts of the global market. As you probably know already, I also like to talk to people and exchange ideas and gain some other peoples’ experience regarding their projects. In addition, I am also an internet addict,
I do admit that – I shop online, read blogs and watch movies on the internet.

In a contrary, as a software tester, I am also quite picky user and when something on the website pisses me off – I don’t use the service anymore, unless I have no other choice.

What I would like to emphasise here is the matter of choice, in fact.

In my short software tester’s career (too many times!) I came across with the approach that when we know about the bug – our job is done. It takes too much effort (MONEY) to fix it, so we collect non-critical ones in our backlog and struggle to have them fix for end users. Does it sound familiar to you, Dear Reader?

It made me cry so many times observing my projects sinking in the sea of spaghetti code and UI issues.  Unfortunately, it was often way too visible, that I had been the only person, who bothered.

And here we are – software testers – doing the best we can to expose flaws of our products,  to protect our users, being angry and powerless at the same time. Product managers, product owners don’t listen to us, they just sell and expand the products worldwide.

Is it bad? It seems to be short-term profitable, but sadly, we may end up with piles of hideous software as we end up with polluted air, rivers and garbage in Indian Ocean.

When talking about the choice – there are branches on the market – which benefit from not having a competition. In Poland, where I live, this would be a public sector’s case – schools, national insurance, security. There is usually a tender in order to choose
a supplier of a software for certain public organisation, but in most of the cases the cheapest offer wins.
How do people manage to obtain the cheapest offer? They don’t include quality in their estimations. Simple.
Who will be the most impacted by the poor quality of the software here? Us. Hopefully, the person who made all bad decisions as well. An employee, who uses this software on daily basis has NO CHOICE. He is forced to use CRAPPY software, because it’s a part of his job. He must struggle, because somebody a few months back – was thinking just about his own profit. This is just against work ethics – but it happens in most of the cases.

We sell more than we have coded

Quality – not quantity – this is the sentence we teach newcomers in testing industry.
However, sometimes I think, that there is literally nobody, who cares about software quality. What’s the point of testing, if we don’t fix our mistakes?
During my last speech at SeeTest I said, “as a result of selling non-existing software” and, frankly, there or wherever I said that – the audience shared same feelings and knew exactly what I was talking about.

Are we in software delivery era, in which quality doesn’t matter anymore?

Out ‘businesses’ sell features, which are in their minds only, claiming it’s already existed and after that push us to write code faster, without impediments. What I mean by distractions – bug fixes for instance. I was in such projects, I know how that feels and how hopeless people are in such crazy circumstances.

We deliver faulty software in enormous pace

What do we have in return?
Faulty websites, online stores which display 503 or 404 on daily basis, dramatic UI, crashing mobile applications or just literally – non-usable software.

I don’t mind fast pace of work – it is stimulating and efficient, but I miss the care about end user. We produce fast-food software rather that slowly cooked pieces which would make us proud. I don’t think that a software tester is glad when his team receives bugs or complaints from production, but how could he help in the first place?

I am angry, because during my online shopping it happens more often that I receive errors, UI issues, drop-downs without items or my mobile applications just crash. When
I want to be kind and nice – I report those issues – but usually nobody responses and the issues  still exist.

We keep on racing each other – neglecting quality. ‘Fast’ is the new black.

Sadly, I don’t think that even persistent testers are able to fight with the approach. Maybe, there has to come a moment, as it was with coal usage or waste in the oceans,  when it will be obvious for us to stop and think.
I hope, there will be this moment, once more, in software industry, when ‘business’ would start to think not only about their fast profit but also about the end user.

Sorry for melancholy today, it just bothers me.

What do you think?
Feel free to comment down below, on Twitter or Facebook. Cheers.

 

 

Posted in exploratory testing, mobile testing, production bug, tools

(Live) slow connection mobile testing

(s)lowconnection (2)

Holidays are these times when we go outside, fly abroad, spend a lot of time on wasting it and basically doing nothing. What is a common feature of such places?
Slow internet connection – no matter if it is within your network or WiFi.
I, an addicted smartphone user, find it as a one of the most annoying things EVER!

As I probably wrote at least once, I use my smartphone (Android) as a mother ship 😀
(It would really be a pain if I lost it).
Anyways, I use my phone for shopping purposes as well. What is more, taking into account a ‘typical woman‘ stereotype – I do a lot of shopping. 90% of which I do online. The conclusion is simple – the more time I have to waste – the more I buy 🙂 Having said that, imagine my frustration during online shops explorations, when I do experience slow internet connection.
In addition, I am not talking about websites only, but about well-known native apps, which , as it turned out, are not supposed to handle limited network range.

Today, I would like to present you a true wall of glory, basing on my holidays explorations. I’ve called it – the SlowCo Art Gallery of inspiring issues.
Don’t get me wrong, I don’t want it to be pure hate only, but maybe an inspiring feature to test for all of you working with any kind of mobile application or websites.

How to handle such testing (to avoid irritating me in the future)? Short introduction down below or in one of my old posts here.

Shall we begin?

Pic.1. Where is my CSS? – Example from douglas.pl

Screenshot_2017-08-12-14-24-30

Pic.2. A journey through classes? – Example from H&M Android app

Screenshot_2017-08-16-09-15-07

Pic.3. We don’t know what the hell just happened, so we’ll provide you with a random message – Example from native Twitter app
(
Translation from Polish: An important service, without which this application is not able to run, is currently not available. Try later.)

Screenshot_2017-08-15-21-02-44

Pic.4. We are creative in producing 404 pages – Example from jednosc.com.pl website

Screenshot_2017-08-11-11-00-31

Pic.5. At least one of those messages is valid. Hopefully. – Example from Google Play app

screenshot_2016-08-12-23-11-04

As you can see – the range of failure possibilities is really vast BUT the issue is the same in each example. In the end, all those bugs may seem funny 🙂

On the other hand, we are able to protect our applications and websites against SHAME by performing sets of basic network connection test cases.

How to stress the app, pretending slow connection?
My favorite option is to open Chrome, click F12 on your computer and go to Chrome Developer Tools.
Navigate to Network -> Throttling and select a connection version that you are interested in. Slow your connection down, and see the magic 🙂
You can also connect your mobile app with Chrome Dev Tools and stress it.

throttling

It may sometimes seem as an edge case, but imagine all the people, who live most of their lives in the area of slow internet connection. How frustrating might it be for them?
Hey mobile testers – Let’s make apps great again! 🙂

Like!, Share, Hate, Comment on Facebook
Stalk me on Twitter
or comment down below 🙂

Cheers!