BlackFriday

Last Friday – I was a hunter – I knew something should go wrong.
But, let’s get back to square one.

My inspiration

Tomek Dubikowski inspired me with his talk “Proper gun makes testing fun” to think more about performance testing, as lack of those would concern any of us in the most surprising situations. Following slide from his presentations would give you very simple guide of what every tester should check, to make sure, that his website or any other service will survive unexpected circumstances.

20171117_162109

This post won’t be a guideline about how to setup your environment in order to test performance of your service – but rather a tour through bad examples and good practices in performance testing.

I am not and expert – I am just gathering knowledge!

Black Friday

Black Friday is quite new term in Poland. It started to gain on popularity a year or two ago, maybe, but it became quite on spot last week. All regular shops in malls and most of online shops with clothing, cosmetics or electronics started 20% discount weekend.
I am a mean tester, so I decided to start my hunt in the evening, when traffic in online shops usually goes high. What did I hunt for? Dramatic errors, of course.

As you can guess, such discounts encouraged more customers to visit online shops and do the shopping. For those, that has not been prepared, it did not ended well. Most websites that I’ve visited were significantly slower than usual, they had some visible issues or problems with loading pictures. Unfortunately, some of them did not pass the load tests at all.

I’ll provide you with three examples of what might happen, if you won’t test performance of your websites, I hope you’ll prepare better for Christmas Sales 😀 (or whatever you wish).

All of them are taken from online stores – so the ‘only’ effects might be the shortage in store’s income and customers’ trust for the brand.

When such bloomer happens to some notional institution – it may end up in the news and in multiple memes. In 2014 we had this situation with counting votes in regional elections. Our government decided to introduce online vote transfer from local committees which ended up with tremendous fail.

As the system was not supporting high loads of transfers (you had this one job), we’ve been waiting for election’s results for a week.

One of my favorite memes from that annoying time – “It works on my machine”:

pkw

Let’s get back to Black Friday issue, shall we?

What can I do

I would divide those performance problems into two groups (both customer – facing):

  1. It doesn’t work and you have a real problem here
  2. Tell it to your customer in some human – readable way

Don’t get me wrong – if that happens to the service of yours – you’re f*** anyway 😀 It depends on you, though, how do you communicate it to your customer and if he would eventually come back to you or not.

If your service doesn’t have performance checked – too many visitors result with too many transactions for it’s capacity, which finally terminates the service with timeout. Reserved.com ‘died ‘without a warning and prompted us with extremely professional ‘504 Gateway Time-out‘.

reserved

Douglas.pl died as well, but somebody had at least done his job on UI beforehand and displayed a user-friendly information ‘We are sorry’. Better – but still…

douglas2

Most of their sites displayed this kind of messages anyway. At least, they tried.

douglas

One of our Polish Facebook Group about testing –  “Testowanie Oprogramowania” member – Artek Zwoliński – spotted fantastic fail as well at X-KOM – online store. X-kom informed us even what kind of database do they use as well 🙂

23926328_2024222767860377_8091906648219521639_o

We may make fun of it, but those examples show real problem with lack of performance testing and lack of effective solutions for peaks and unusual loads on websites. On the one hand, brand owners may claim that those are unique situations within the year and performance testing is too expensive to do it, on the other hand it is just risky for the brand and customer’s trust.

But how?

Performance testing becomes quite trendy thing now, so you can easily use online courses or watch some tutorials about how to get along with JMeter or Gatling.  However, these are just the tools. It is you – who should come up with ideas and foresee the need of testing performance. Do the risk analysis with your business and try to convince them about the need and benefits.

In addition, do your homework and don’t be sloppy in preparations. Equip yourself with knowledge and prepare (buy) an isolated environment, that will be 100% accurate to your production. If not – performance testing does not have sense, sorry. No matter how much will you invest in learning JMeter’s functions (it is already old-fashioned tool BTW).

Isolated environment is the key to successful performance testing, otherwise, happy with meaningless results, you’ll be testing your service live with end users, as many online shops did on Black Friday. 🙂

We, as testers, must be aware of such risks and know what might happen if we skip performance testing part, especially, when we develop a service, which has even a slight chance to be on top for some reason (sale, promotion, unique product etc).

Last, but not least, preserve any results of performance testing, as it may be useful in the future. The best idea is to store all the results directly in the database (for example in MongoDB) in order to have it accessible and ready for analysis.

Cheat sheet of performance testing

  1. Know your PURPOSE
  2. Isolate the environment
  3. Environment must be 100% same with production
  4. Optimise the configuration
  5. Store data
  6. Repeat the test at least once
  7. Get professionals to analyze your results

Yeah, I should get some shiny table for that 🙂

I hope it was useful and will guide you a little more into performance testing.
In case of any comments – let me know down below or stalk me on Twitter of Facebook.

Cheers!

Advertisements

Black Friday – the best performance live testing ever

Posted in conferences

TestWarez 2017 – Complexity

I’m back 🙂

I’ve spent last two days in Toruń getting as much from the best known Polish QA conference – TestWarez – as possible. Each time I take part in such event – I feel like home. People, who have similar mindset, who want to change the world and improve their skills, vivid atmosphere, rush, noise and loads of coffee. All at once and each one separately.

It was my first time at this event, I had some expectations, but the reality was different. Let’s face the truth – TestWarez is great at the point where you can meet people and talk to them, but it has nothing to do with modern worldwide trends in software testing. When, at the same time, at Agile Testing Days in Potsdam speakers talk about exploratory, supporting women (#SupportAfganGirlsRoboticsTeam) and testing web services – TestWarez’es agenda provided us with such innovative ideas like “there are tools more advanced that Excel to report your bugs” (psssst – it is no longer a Stone Age) or “manual tester/automation tester” (15 kittens died during that presentation, if you know what I mean @MichaelBolton).

Don’t get me wrong – it is not about playing down the conference, but maybe it’s time to move on and look around? Maybe, it would be good to see that there is a world out there beyond ISTQB certification – full of fresh ideas how to improve teamwork.

There were some brilliant speeches as well, but they were rather very good talks than innovative ones. Sadly for me, the more I attend conferences – the more I expect – and maybe it’s not the point. I think SJSI – the main organizer – missed the boat in delivering value instead of package. Maybe it’s time to introduce English – only track (if not the whole event) and mark it in the agenda. It’s a shame when foreign guests are not able to benefit from the event as well due to language barrier.

On the other hand – we have such brilliant events in Poland like TestFest or Quality Excites that are alive,  creative and give new energy. In addition,  maybe the events, that don’t cost an arm and a leg, base on true stories and “we can do it” approach, create more value and QA spirit.

Nevertheless, I had great time in unique surrounding of Toruń – old Polish city. I get together with my friends from testing community, talked for hours with testers from all over the country and enjoyed the event a lot.

So – back to square one – my top 5 speeches (and one discussion panel) – from what I’ve selected during the conference. You should definitely look for them, as soon as they emerge on TestWarez YouTube channel.

  1. O sile optymizmu oraz zwinnym rozwoju osobistym – Jędrzej Osiński

It was not exactly about testing, but rather about personal development in general. Light weight presentations, with well-balanced amount of examples made me re-think my life choices any my priorities in life. Very inspiring and pretty fun! My list of books-to-read widened a lot since Friday 😀 Thank you @dr_hawaii

20171117_153310

2. ZEN testów wydajnościowych – Jakub Chabik

There was a lot at TestWarez about performance testing. It seems – this subject is getting trendy nowadays. When our applications run in production quite well – all we have to do is stress them and check how many users can we serve at once. This presentations gave me the receipt how to start, how to manage the environment and which mistakes to avoid since the beginning of my performance testing. Well organized speech – original ZEN- related surrounding – well done!

20171117_110100

3. A proper gun makes testing fun – Tomasz Dubikowski

It may be the first time when Tomek’s speech is not on the top of my list 🙂
The talk was fun as always. Tomek’s jokes, minions and colorful slides shall provide you with all you need from a good speech. He was talking about performance testing as well, gave some epic fails examples and coded live (successfully) using Gatling. I hope we’ll have the opportunity to see it live again on some other event.

20171117_160741

20171117_162109

4. What tester can learn in support – Maciej Wyrodek

This talk was a story about Maciek’s journey as a software tester and it was focused on his first job. He had a lot to do with a support of his product – not only with testing. Below slide summaries this job perfectly.
Testing is not the end – support is!
Maciek’s talk was entertaining, as he used (my) trick with candies 😉 He played a game with the audience, so nobody got bored. The talk was in English – so once it’s on YT – all of you can hear the story.

20171116_165623

5. Jak zaplanować testy, żeby nie wylądować w czarnej d…ziurze – Łukasz Pietrucha. – discussion panel

I can remember when Łukasz hosted first WrotQA (local testers meetups in Wrocław – the city I live in) meetings. It was long time ago in a galaxy far far away. It was a time when I wore diapers as a software tester 🙂

Today, he is a storyteller and a professional speaker. As I wrote about the discussion panel itself in my previous post – I have to admit that I’m impressed by the talk itself. We had an opportunity to take part in moderated discussion at professional level. People were truly involved and took some examples fro themselves, I believe.

I’ve enjoyed it a lot 🙂

20171116_173325

6. Przychodzi tester na rozmowę – Patryk Hemperek

The dilemma was big – Patryk and Kamila Mrozek (my ‘homies’ from Worcław) had their presentations at the same time (come oooon TestWarez!). As I saw Kamila in action before – I decided to support Patryk at his speech about evolving as a software tester. He was talking about his journey and experiences as a software tester and focused on gaining new skill to improve test automation in his project. Very instructive talk –
I recommend it especially to all of you who would like to start their journey as a software tester.

20171117_131112

 

I wish I could see more – but I was the only one among 5 (!!!) tracks at once. There was some about test automation, lot about performance testing and even more about ISTQB – related stuff. I hope I’ll see more online.

… And one more thing – 4 – in my opinion the most tempting presentations – were scheduled during the last slot on Friday. 70% of the conference attendees had left before the speeches started 😦 It made me sad. It is horrible to talk to the empty room. It is also horrible to give a great talk that no one listens to.
Re-think it, please – both organisers and attendees.

What did you like the most about Test Warez?
Was my summary helpful?

As usual – don’t hesitate to comment down below or on Twitter / Facebook.

Cheers!

Posted in agile, conferences, team, Uncategorized

Test team will help you out

Test Team

Hi Boys and Girls,

Being close to the test community at Test Warez Conference, on which I am at the moment, made me think about my team and how do we do things at New Voice Media.
I believe it is worth to spread and inspire you to introduce good practices into your test / scrum teams.

Next week I’ll provide you with wider summary of Test Warez – today I want to focus on one aspect that came to my mind yesterday during the discussion panel run by
Łukasz Pietrucha about planning your tests.

20171116_173325
We’ve started with lightweight talk about ISTQB’ish approach to formal documentation and planning across organisations but end up with vivid discussion and sharing good practices and personal project-related experiences (not necessarily related to panel’s subject 🙂 ). It made me think that planning your tests and organising testing in your organisations, in general, is extremely context-related. You might think that one, structured, recommended by ISTQB idea should work, but sometimes, to be honest, it is just useless.

I went back to my roots as a software tester.

One of my very first sources of knowledge about software testing in general was Polish blog.testowka.pl . It is technical, teaches you how to start with Selenium and gives updates about software testing in general – very thought through source of knowledge (For some reason I was convinced that it is run by a girl…. but never mind, just leave it 🙂 – sorry Wiktor! ).
Wiktor Żołnowski – the author – wrote a few words about himself on that blog. However, I’ve read it just a week or two ago. Wiktor wrote ‘It was ‘Agile’ – people and interactions over processes and tools. Then I’ve acknowledged that all things which I knew about testing ans so-called quality processes promoted by different organisations, had little value. Software can be crafted just better.‘ – and I consider it as a quote close to my heart. I still keep thinking about it, that’s why I decided to write today’s post.
Now, at New Voice Media, I can tell the same thing. There was always a missing part in my teams / projects/ organisations, even with their structured processes and diverse working environments, and I don’t speak about faking the agile style of work only – what I mean is – craftsmanship and team spirit (what a cliche).
It suites me better – it may not suit you at all, so don’t feel offended, Dear Reader 🙂

k0lN1UVoMn64u8b

As some of you probably know, at New Voice Media we – the DevOps team – work  in Scrum or Scrumban. This is the first time, where I an able to see theory in practice and it works good for the organisation. We have testers and developers in our team, but we try to widen our responsibilities to enable all team members to learn and improve their skills.

Apart from separated feature teams – we also try to gather in community of interests, Sound ‘Spotify’ish’ 🙂 Maybe. On the other hand, it helps. We try to share knowledge across teams and locations (some of us work in Poland and some in the UK) to avoid silos of knowledge

Ralls_Texas_Grain_Silos_2010

– it means we meet, talk and help one another out. It also means, that when somebody gets sick, has some emergency or has too many tasks to do at once – other testers may (and will!) help. We use different communication tools, chatters, video conferences, Wiki spaces and so on, but first of all – WE WANT to share and WE WANT to learn. It is not the organisation, who makes us do it – it’s us who do it, because it just helps.

I am not sure if that would be an approach for entire corporation – but for small departments – maybe? Would it work in a software house? I don’t know – but at leas you may try it. I know at leas one software house, that has it’s own community of interests and it works great for them. 😉 When you feel the energy and willingness to do something – you can definitely progress at things.

You cannot build (good) your software alone these days, so it is good to have a team which would help you out. Just in case 🙂

As always, you can comment down below or stalk me on Twitter.

Cheers!