To tell somebody that he is wrong is called criticism. To do so officially is called testing.
It’s been a while since I’ve posted something. Honest apologies for everyone, who visited my blog looking for some updates and found none.
Blog and I – we were on a break 😉
Rehearsals for Musical “Freedom” consume most of my spare time, but the premiere is getting closer. (fingers crossed!) Tickets will be available soon – so for now save the date:
April 25th / 26th 2018 – Teatr Polski – Wrocław (Poland)
We are collecting money for Fundacja Jaśka Meli Poza Horyzonty.
As a part of this project, last week, I had a chance to SING IN PUBLIC 🙂 It was extremely thrilling experience, but gave me a lot of power for further work.
There is also a recording from my performance. I was hesitating if I should publish it, but after all we do it for people in need. SO – you can watch the video – BUT – you have to promise me – you’ll donate the foundation after you watch it. DEAL?
Last Friday – I was a hunter – I knew something should go wrong.
But, let’s get back to square one.
Tomek Dubikowski inspired me with his talk “Proper gun makes testing fun” to think more about performance testing, as lack of those would concern any of us in the most surprising situations. Following slide from his presentations would give you very simple guide of what every tester should check, to make sure, that his website or any other service will survive unexpected circumstances.
This post won’t be a guideline about how to setup your environment in order to test performance of your service – but rather a tour through bad examples and good practices in performance testing.
I am not and expert – I am just gathering knowledge!
Black Friday is quite new term in Poland. It started to gain on popularity a year or two ago, maybe, but it became quite on spot last week. All regular shops in malls and most of online shops with clothing, cosmetics or electronics started 20% discount weekend.
I am a mean tester, so I decided to start my hunt in the evening, when traffic in online shops usually goes high. What did I hunt for? Dramatic errors, of course.
As you can guess, such discounts encouraged more customers to visit online shops and do the shopping. For those, that has not been prepared, it did not ended well. Most websites that I’ve visited were significantly slower than usual, they had some visible issues or problems with loading pictures. Unfortunately, some of them did not pass the load tests at all.
I’ll provide you with three examples of what might happen, if you won’t test performance of your websites, I hope you’ll prepare better for Christmas Sales 😀 (or whatever you wish).
All of them are taken from online stores – so the ‘only’ effects might be the shortage in store’s income and customers’ trust for the brand.
When such bloomer happens to some notional institution – it may end up in the news and in multiple memes. In 2014 we had this situation with counting votes in regional elections. Our government decided to introduce online vote transfer from local committees which ended up with tremendous fail.
As the system was not supporting high loads of transfers (you had this one job), we’ve been waiting for election’s results for a week.
One of my favorite memes from that annoying time – “It works on my machine”:
Let’s get back to Black Friday issue, shall we?
What can I do
I would divide those performance problems into two groups (both customer – facing):
It doesn’t work and you have a real problem here
Tell it to your customer in some human – readable way
Don’t get me wrong – if that happens to the service of yours – you’re f*** anyway 😀 It depends on you, though, how do you communicate it to your customer and if he would eventually come back to you or not.
If your service doesn’t have performance checked – too many visitors result with too many transactions for it’s capacity, which finally terminates the service with timeout. Reserved.com ‘died ‘without a warning and prompted us with extremely professional ‘504 Gateway Time-out‘.
Douglas.pl died as well, but somebody had at least done his job on UI beforehand and displayed a user-friendly information ‘We are sorry’. Better – but still…
Most of their sites displayed this kind of messages anyway. At least, they tried.
One of our Polish Facebook Group about testing – “Testowanie Oprogramowania” member – Artek Zwoliński – spotted fantastic fail as well at X-KOM – online store. X-kom informed us even what kind of database do they use as well 🙂
We may make fun of it, but those examples show real problem with lack of performance testing and lack of effective solutions for peaks and unusual loads on websites. On the one hand, brand owners may claim that those are unique situations within the year and performance testing is too expensive to do it, on the other hand it is just risky for the brand and customer’s trust.
Performance testing becomes quite trendy thing now, so you can easily use online courses or watch some tutorials about how to get along with JMeter or Gatling. However, these are just the tools. It is you – who should come up with ideas and foresee the need of testing performance. Do the risk analysis with your business and try to convince them about the need and benefits.
In addition, do your homework and don’t be sloppy in preparations. Equip yourself with knowledge and prepare (buy) an isolated environment, that will be 100% accurate to your production. If not – performance testing does not have sense, sorry. No matter how much will you invest in learning JMeter’s functions (it is already old-fashioned tool BTW).
Isolated environment is the keyto successful performance testing, otherwise, happy with meaningless results, you’ll be testing your service live with end users, as many online shops did on Black Friday. 🙂
We, as testers, must be aware of such risks and know what might happen if we skip performance testing part, especially, when we develop a service, which has even a slight chance to be on top for some reason (sale, promotion, unique product etc).
Last, but not least, preserve any results of performance testing, as it may be useful in the future. The best idea is to store all the results directly in the database (for example in MongoDB) in order to have it accessible and ready for analysis.
Cheat sheet of performance testing
Know your PURPOSE
Isolate the environment
Environment must be 100% same with production
Optimise the configuration
Repeat the test at least once
Get professionals to analyze your results
Yeah, I should get some shiny table for that 🙂
I hope it was useful and will guide you a little more into performance testing.
In case of any comments – let me know down below or stalk me on Twitter of Facebook.
I’ve spent last two days in Toruń getting as much from the best known Polish QA conference – TestWarez – as possible. Each time I take part in such event – I feel like home. People, who have similar mindset, who want to change the world and improve their skills, vivid atmosphere, rush, noise and loads of coffee. All at once and each one separately.
It was my first time at this event, I had some expectations, but the reality was different. Let’s face the truth – TestWarez is great at the point where you can meet people and talk to them, but it has nothing to do with modern worldwide trends in software testing. When, at the same time, at Agile Testing Days in Potsdam speakers talk about exploratory, supporting women (#SupportAfganGirlsRoboticsTeam) and testing web services – TestWarez’es agenda provided us with such innovative ideas like “there are tools more advanced that Excel to report your bugs” (psssst – it is no longer a Stone Age) or “manual tester/automation tester” (15 kittens died during that presentation, if you know what I mean @MichaelBolton).
Don’t get me wrong – it is not about playing down the conference, but maybe it’s time to move on and look around? Maybe, it would be good to see that there is a world out there beyond ISTQB certification – full of fresh ideas how to improve teamwork.
There were some brilliant speeches as well, but they were rather very good talks than innovative ones. Sadly for me, the more I attend conferences – the more I expect – and maybe it’s not the point. I think SJSI – the main organizer – missed the boat in delivering value instead of package. Maybe it’s time to introduce English – only track (if not the whole event) and mark it in the agenda. It’s a shame when foreign guests are not able to benefit from the event as well due to language barrier.
On the other hand – we have such brilliant events in Poland like TestFest or Quality Excites that are alive, creative and give new energy. In addition, maybe the events, that don’t cost an arm and a leg, base on true stories and “we can do it” approach, create more value and QA spirit.
Nevertheless, I had great time in unique surrounding of Toruń – old Polish city. I get together with my friends from testing community, talked for hours with testers from all over the country and enjoyed the event a lot.
So – back to square one – my top 5 speeches (and one discussion panel) – from what I’ve selected during the conference. You should definitely look for them, as soon as they emerge on TestWarez YouTube channel.
O sile optymizmu oraz zwinnym rozwoju osobistym – Jędrzej Osiński
It was not exactly about testing, but rather about personal development in general. Light weight presentations, with well-balanced amount of examples made me re-think my life choices any my priorities in life. Very inspiring and pretty fun! My list of books-to-read widened a lot since Friday 😀 Thank you @dr_hawaii
2. ZEN testów wydajnościowych – Jakub Chabik
There was a lot at TestWarez about performance testing. It seems – this subject is getting trendy nowadays. When our applications run in production quite well – all we have to do is stress them and check how many users can we serve at once. This presentations gave me the receipt how to start, how to manage the environment and which mistakes to avoid since the beginning of my performance testing. Well organized speech – original ZEN- related surrounding – well done!
3. A proper gun makes testing fun – Tomasz Dubikowski
It may be the first time when Tomek’s speech is not on the top of my list 🙂
The talk was fun as always. Tomek’s jokes, minions and colorful slides shall provide you with all you need from a good speech. He was talking about performance testing as well, gave some epic fails examples and coded live (successfully) using Gatling. I hope we’ll have the opportunity to see it live again on some other event.
4. What tester can learn in support – Maciej Wyrodek
This talk was a story about Maciek’s journey as a software tester and it was focused on his first job. He had a lot to do with a support of his product – not only with testing. Below slide summaries this job perfectly. Testing is not the end – support is!
Maciek’s talk was entertaining, as he used (my) trick with candies 😉 He played a game with the audience, so nobody got bored. The talk was in English – so once it’s on YT – all of you can hear the story.
5. Jak zaplanować testy, żeby nie wylądować w czarnej d…ziurze – Łukasz Pietrucha. – discussion panel
I can remember when Łukasz hosted first WrotQA (local testers meetups in Wrocław – the city I live in) meetings. It was long time ago in a galaxy far far away. It was a time when I wore diapers as a software tester 🙂
Today, he is a storyteller and a professional speaker. As I wrote about the discussion panel itself in my previous post – I have to admit that I’m impressed by the talk itself. We had an opportunity to take part in moderated discussion at professional level. People were truly involved and took some examples fro themselves, I believe.
I’ve enjoyed it a lot 🙂
6. Przychodzi tester na rozmowę – Patryk Hemperek
The dilemma was big – Patryk and Kamila Mrozek (my ‘homies’ from Worcław) had their presentations at the same time (come oooon TestWarez!). As I saw Kamila in action before – I decided to support Patryk at his speech about evolving as a software tester. He was talking about his journey and experiences as a software tester and focused on gaining new skill to improve test automation in his project. Very instructive talk –
I recommend it especially to all of you who would like to start their journey as a software tester.
I wish I could see more – but I was the only one among 5 (!!!) tracks at once. There was some about test automation, lot about performance testing and even more about ISTQB – related stuff. I hope I’ll see more online.
… And one more thing – 4 – in my opinion the most tempting presentations – were scheduled during the last slot on Friday. 70% of the conference attendees had left before the speeches started 😦 It made me sad. It is horrible to talk to the empty room. It is also horrible to give a great talk that no one listens to.
Re-think it, please – both organisers and attendees.
What did you like the most about Test Warez?
Was my summary helpful?
As usual – don’t hesitate to comment down below or on Twitter / Facebook.