Let's talk about testing

Standard platform, operating system and browser recommendations (Sept 16)

This entry is part 10 of 10 in the series Standard Platform, Operating System and Browser Recommendations


The following post updates our recommendations for platforms, operating systems and browsers to use when testing commercial web sites targeting consumers in Europe.

Our recommendations are based on usage figures widely available on the Internet, our experience and our analysis of client needs. Since browser versions change frequently, we review and update these recommendations regularly.

Continue Reading »

The zombie Scrum apocalypse is upon us!

A couple of weeks ago, I met them. In real life. Scrum zombies! It was at Scrum day Europe 2016. Scrum day is not their natural habitat, they were brought in by two highly skilled Scrum zombie fighters as examples of what can happen when you are touched by the Scrum zombie virus. The zombies were examples shown in the workshop, ‘How do we survive the zombie Scrum apocalypse’ by Johannes Schartau and Christiaan Verwijs. I am going to summarize their workshop and then elaborate a bit more of what I think is the role of the tester in fighting the zombie Scrum apocalypse. For a more detailed view on zombie Scrum, please check their blog.

What exactly is the zombie Scrum apocalypse? And more important; am I already a zombie, maybe even without knowing it? The main problem with zombie Scrum is that it looks like Scrum; the team consists of a Product Owner, Scrum master and a Development Team. They follow the Scrum events; they work in sprints, do sprint plannings, have a daily scrum, sprint review and a sprint retrospective. They have the Scrum artifacts. But they are missing out on the purpose of their work; they don’t seem to deliver working software at the end of the sprint. They miss out on what is the core of Scrum.

Of course, there are other symptoms. Zombie Scrum teams desire no contact with the outside world; they don’t want to get involved with their customers, they don’t care if their poorly built application breaks the end-users machine. They did their job; delivering code. QA failed to catch the bugs, design failed to create a nice look. Not their fault! Whether or not their sprint succeeds or fails, the zombie Scrum team is not affected by it. They don’t try to improve, they just sink in the swamp they created for themselves. They become numb. In a worst case scenario, the Product Owner is absent most of the time, the Scrum master has multiple teams to take care of, developers are often being swapped between teams, so there is no stability. Zombie Scrum teams go through the motions of Scrum, but miss out on the fun in Scrum.

There are many causes of zombie Scrum. Probably the most well known cause (I guess we have seen it all) is the company that want to adapt this fancy new way of developing working software (#hype!). They rename the project manager to Scrum master, start to work in sprints, call the development team Scrum team and there you have it: the company is Scrum. The team slips in a non-productive way of acting-Scrum and starts showing above mentioned symptoms.
Another cause may be the lack of urgency zombie Scrum teams feel. They don’t feel the urgency to connect with their customers to deliver valued, working software. They don’t feel the urgency to learn from their mistakes. Every situation is different however, so take a moment to think about other possible causes of zombie Scrum.

So, having read all this and getting a little depressed by the fact you recognise some of the symptoms; what are we going to do about his? And, for me as a tester, what can I do? How can I contribute to healthy Scrum? As a tester, you are usually working in close collaboration with the developers and the product owner. Talk to them about quality and the importance of delivering working software. Help the product owner by testing his requirements beforehand. Clear requirements will help the developers build working software within the sprint.
Personally, I think the most important way of improving the way your Scrum team works is by talking about what you are building and how you are building it. Talk about your definition of ‘done’. If your team doesn’t have one, try to establish one. Make sure quality assurance is an important part of it. When you are testing, make sure to check if the definition of done is met. If not, discuss your findings with your developers; how can we make sure the software meets the definition of done and we can meet the goal of Scrum; deliver working software every sprint.

There are more ways of improving your Scrum team as a tester, but most of them come down to communication. Communicate with your team about the process and about how you contribute to the sprint goal. Raise your voice in the retrospective. Talk about the purpose of Scrum and make sure you are contributing to it.

If you think you are in a zombie Scrum team, don’t get depressed. Show your team that when you improve, there is fun in Scrum!

Yahoo no longer does software testing; continuous integration and what it means to software testing

Recently I saw on one of Linkedin’s software testing groups a post that said Yahoo no longer does software testing. The headline was meant to grab the attention of the group in a controversial way so as to guarantee a lot of discussion. And, it did. So what do they do? I asked as I started reading the article. It turns out that Yahoo now do continuous integration and continuous deployment (CI/CD) instead of software testing. The job of testing is now done by developers writing unit tests in a DevOps process and formal testing is no longer a specific phase or activity in their sprint planning. Clearly this is bullshit. No company as hardcore tech as Yahoo would release anything to production without it undergoing some type of testing. After a few exchanges with the original author, he did admit that Yahoo still do testing, but it’s not done at the end of a sprint, it’s done all the time as part of their hyper fantastic CI/CD process.

This isn’t the first time that I had heard someone evangelising CI/CD like it was some new magic bullet that could solve all the world’s software release problems. Nor the first time someone had suggested CI/CD does away with the need for software testing. But does it really, when you get right down to it?

CI/CD done as part of a full devops process is really nothing more than a very rapid development cycle measured in hours. The people involved can wear multiple technical hats including business analyst, developer, and sysadmin. However as a methodology it doesn’t prevent bugs getting into software. But it does make it fast to identify them, if you have the right people and process. Testers can and do work in a CI/CD environment. They are multi-skilled individuals capable of undertaking manual functional testing on new features that come straight out of development, and building automated test suites once the features have stabilised. These testers will be involved alongside the other core technical team from day-1 of whatever project plan in place. But they aren’t your grandfather’s software testers.

Clearly the role of web services software testing is evolving as software development methodologies evolve, as do the people that take on the job. 5 years ago functional testing was a job unto itself, and you could make a career as a functional tester without ever having to touch a line of code (let alone write one yourself). But software development is more and more geared towards: faster-better-cheaper. The internet and the always-on-demand culture it has created means something new has to be released very nearly daily, or your site visitors lose interest. The most in-demand test professionals will be the ones that integrate with a development team and have some overlap with them. This means, having the ability to talk tech with the DevOps team and understand what they are doing, setting up automated test tools from internet-download to deployment on the test environment directly, (maybe even set up the test environment if needs be), perform all the testing work that is the job and do it all to high levels of industry best practice standards. This new test professional will be technical themselves and most likely have spent time in the trenches of a software project as a developer or sysadmin, and understands technical architectures and environments.

Does this mean the end of the pure functional tester? Probably not, but I believe over time it will become a niche role better suited to a big team with tightly boxed areas of responsibility where one person does one thing only. For the tester working in a small team they will most definitely need to be a jack-of-all-trades, with mastery to be learned as required. This is why we are now seeing such job advertisements as Developer-in-test which definitely didn’t exist 5 years ago.

Which brings me back – finally – to Yahoo. You know that these guys know software development, and for sure they will be using the very best process known today. That they are using CI/CD to get the most amount of speed possible for new feature releases does not mean they are not testing. The CI/CD DevOps process is all about continuous testing, rather than the absence of testing. As code is checked-in to the code base, testing is triggered, and bugs will be found, reported, and fed back into development. It will all look very seamless and on a project plan the absence of a test phase will be very noticeable, but you can be sure testing is absolutely being done.

Standard platform, operating system and browser recommendations (June 16)


The following post updates our recommendations for platforms, operating systems and browsers to use when testing commercial web sites targeting consumers in Europe.

Our recommendations are based on usage figures widely available on the Internet, our experience and analysis of client needs. Since browser versions change frequently, we review and update these recommendations regularly.

Continue Reading »

Cloud bullying

This entry is part 4 of 4 in the series Recruitment

My alien has returned to his own planet, leaving me with a re-organized brain and a manual. That gives me enough space upstairs to think about strategy, namely: why would a brilliant tester want to work at spriteCloud? Good question!

In search of some inspiration for my weekly blog, the morning news brought an interesting article; bullying at work is a costly affair and more than one million people are victims of some kind of bullying. How sad is that? I personally think the real figures are much higher, more of an iceberg effect.

Bullying is costing our society and organizations billions, because people call in sick much more, and are less productive, because they are less motivated. On top of that, managers are reason number one why people leave an organization. That is the other cause of a huge amount of extra costs.

So much for the process and cost optimization in those companies. I figure much of those managers are bullies themselves. We at spriteCloud see that as personal and professional incompetence and that is why we don’t have managers. Just our board of the three musketeers, the owners and also our brilliant testers themselves. Of course we make fun of each other at times, but we respect each other and motivate and stimulate each other where necessary and we never have to ask to get some help.

That’s why after my Suits disaster last week, I could watch the complete series last weekend, thanks to my dear colleague Stephan, our financial controller. He has promised me the rest of the series as well. (I won’t be available for any social activities for the next couple of weeks.) You see, that is what we call good fellowship and that is why we love working here; we don’t bully, we communicate.

Could the above be reasons why testers want to work at spriteCloud?

Let me know what you think.

Which Android versions should I test on?

This entry is part 2 of 2 in the series Android

Another month (and a bit) has passed, so Google has published new Android statistics []. There are no major changes but the trend is clearly visible. Only the latest Android version with market share (currently Android 6.0) is increasing. All other versions are stable or losing market share. This leads me to the next valuable question about existing versions.

Q: How long will it take before a new Android version gets at least 5% market share?

The reason I want this question answered is because when it goes above the 5% marker, it’s significant enough to be worth testing. At least until the moment a new version is released. Before we answer the question let’s look at the current Android market share numbers.

Results from the graph interpreted:

  • Android 6.0 Marshmallow is growing rapidly and has 7.5% market share
  • Android 5.1 Lollipop is stable this month
  • All other versions (5.0 and lower) are losing market share
  • Android 4.4 remains the largest Android version, followed by the Lollipop 5.1 & 5.0
  • Android 4.1 and 4.2 are also still worth testing
  • Android 4.3, 4.0, 2.3 & 2.2 are debatable if they still need to be tested
  • I would not test 2.2, 2.3 & 4.0 anymore unless these versions still generate revenue for you
  • For version 4.3, I would only perform a smoke test to see if it’s usable

Now back to the question “how long will it take before a new Android version gets at least 5% Android market share?”

I started collecting statistics in October 2012. This means versions older than Android 4.2 didn’t register. Also keep in mind that the release date of a version is not in sync with the statistical collection date. Therefore I am only able to make a rough estimate – but even so, a trend is visible.

  • Android 4.2 took about 8 months to get above the 5% market share line
  • Android 4.3, 4.4, 5.0 and 5.1 all took around 5 months
  • Android 6.0 took about 7 months

Conclusion: Roughly six months after the release date, the new version will gain around 5% of the Android pie.

I also notice that one or two months after a new Android version exceeds 5% market share, all older versions lose market share. The only exceptions so far have been the Jelly Bean releases. This appears to be because the releases were close to each other (within less than 6 months).

Lastly, I am aware that only a selection of high-end devices receive the latest Android versions after they have been purchased. Mid-range or low-end devices often do not get upgrades. As long as this remains the case, I expect the trends I identify above to continue.

Any questions?

Cloud organization

This entry is part 3 of 4 in the series Recruitment

My third week and something upstairs is happening. It feels like something is sorting and rearranging all information. A bit like the alien in “Men in Black II” in the post office. But because it feels like I outsourced it to an alien as well, there is a strange feeling going on between starting to make some sense now, but still no idea where in my head that came from. I hope the alien doesn’t forget to leave a user manual behind!

Anyway, week three and some rhythm is kicking in, but writing a blog in this state of (dis)organization is a different story.

Usually enthusiasm, initiative and spontaneity are my most valued characteristics, but right now a reorganizing alien in my personal cloud upstairs is giving me a writer’s block. You all remember that moment where you really wished that some of your characteristics weren’t so dominant? Mine was my enthusiastic and spontaneous “YES!” Now what?

I really love the series Suits. I am a big fan of Harvey and Louis (and Donna, I love Donna), so I installed myself in weekend mode on the couch and there was my inspiration. I always record the episodes, because I hate commercials. I hit the button for record and … nothing happened, the hard drive got stuck. Not the first time, it happens regularly lately – with the provider who claims to have been chosen the best provider in the Netherlands. Now I’m starting to wonder.

The thing is, you lose customers if you don’t have your services in place. For me this is a deal breaker and this example shows how important it is to test your software. Time to transfer to another provider. Hopefully they do test their software and I can watch Harvey anytime I want.

What are your most valued characteristics as a tester?

Cloud party

This entry is part 2 of 4 in the series Recruitment

I survived my first week and let me tell you my head was filled with information. Best to be described as complete chaos upstairs. You know that feeling when you know that you know, but you have no idea where to find it? It is somewhere in that grey mass and hidden between all other information, but at this stage there is no logic to finding it.

Luckily we had a long weekend and I thought that would give me the chance to clear that chaos inside my brain, but it spiralled into worse. During eating, sleeping and while breathing, ideas tried to access my chaos. Questions wanted to join the party upstairs as well. They all succeeded. Panic came over me – how am I going to find new colleagues who fit within this diverse bunch, with the absolute demand that they want to be the best, because we want to be the best? So, don’t ask me how my weekend was, because I don’t remember. All memory space was already taken by the chaos party, no space left.

I need to have a strategy, because I have to compete with all the other brilliant recruiters who know everything there is to know about bug testing and use Big Data to find you (yeah, I hear you laugh, you testers out there).

The thing is, I have no prior knowledge of bug testing and asking around a bit, the general opinion of “recruiters” isn’t really positive either, putting it mildly.

My ‘Eureka!’ moment came during my first company meeting with pizza and beer. I got the chance to meet all my colleagues together and listen to their updates and stories, and experience the positive atmosphere. Being part of this bunch of people is actually a kind of special, because they all have a story to tell that is out of the ordinary.

And we love being extraordinary! How about you? Let me know!

We eat, sleep, live, breathe bug testing

This entry is part 1 of 4 in the series Recruitment

So I was told when applying for the position of IT recruiter at spriteCloud in Amsterdam.

But first, let’s go back to the beginning. That was the moment I decided to give up my independence and commit myself to a company as an employee. Now, being a bit of a rebel, the challenge was to find a company who actually likes that part of my personality in combination of course with my endless wisdom and knowledge of recruitment 😉

Recruitment is my passion, but that only works in the right environment that suits my personality. And then I saw the vacancy and I remember telling the consultant after she told me about these people, that she was describing my job. It was, because it’s mine now.

When I started last Monday, I immediately felt welcome. At first glance you might think this bunch of people have nothing in common and were picked at random. They are totally different from each other. Then my boss Martin came in with a large box and opened it, there was the most beautiful cake I have ever seen. It was made by Leontine in her free weekend especially for me and let me tell you, I am quite picky, but this was the best cake I have ever tasted. We all agreed that she would be the winner of next season “heel Holland bakt”. That would never happen, was her firm reply. (Lucky contenders, they would have otherwise never been able to win.) So we decided we make the cake one of our best USP’s. The cake alone would be reason enough to want to work at spriteCloud.

And then I got it, after talking to my new colleagues. We embrace diversity and we accept and respect each other for who we are, with all the issues and bugs all of us possess and boy, do we have bugs. One person’s bug is the another one’s talent and what bonds and binds us is our obsessive drive for quality and wanting to be the best in what we do, always!

We mix that with good fellowship, humour, passion and the love for good food. E.g. Tuesday is international soup day – we try to soup from around the world.

So now I understand what “we eat, sleep, live and breathe bug testing” means!

Oh, by the way, you bug testers out there, you are welcome to challenge Leontine’s testing skills any time or Andy’s, Mark’s, Martin’s, Daniel’s, Gijs’s, Carlos’s, Claudia’s, Mark’s, Roelof’s, Rubeena’s, Wilco’s or Vamsi’s for that matter. You win? You get your own cake.

Talk to you next week!

Why game testing is important

This entry is part 2 of 2 in the series Game Testing

During the first part of my discussion I covered part of the history and current market value of game and mobile game testing, as well as the simple differences it holds compared to application and web testing. In this second part I would like to outline in more detail how game testing is executed.

Continue Reading »

Reputation. Meet spriteCloud

Find out today why startups, SMBs, enterprises, brands, digital agencies, e-commerce, and mobile clients turn to spriteCloud to help improve their customer experiences. And their reputation. With complete range of QA services, we provide a full service that includes test planning, functional testing, test automation, performance testing, consultancy, mobile testing, and security testing. We even have a test lab — open to all our clients to use — with a full range of devices and platforms.

Discover how our process can boost your reputation.

Optimization WordPress Plugins & Solutions by W3 EDGE