Let's talk about testing

Put Software Quality Assurance in it’s Place!

Here at spriteCloud HQ, we all share the same background story. Details vary wildly, of course, but some or all of the following holds true for each one of us:

We’ve all worked at companies where software quality assurance didn’t exist. We’ve worked at companies where it didn’t exist, yet some people maintained it did. We’ve worked at companies where SQA was an afterthought, and consequently failed. We’ve worked at companies where that was taken as a reason not to bother with SQA. We’ve worked at companies where people didn’t even know what SQA was. We’ve worked at companies who said “our developers can do QA”, or “we get free QA from our beta users”, and whose software was then ripped apart by users for its bad performance or lack of stability.

We’ve also worked at companies where SQA was taken seriously and executed well, but for most of us those were few and far between.

That begs the question: why is that the case?

More precisely, why is it that a multi-billion dollar industry survives and thrives when it systematically neglects an aspect of product development that no other industry can afford to neglect?

Now let me qualify that previous statement: the big players, the likes of Amazon, Google, Facebook, etc. — they take SQA seriously. I would go as far and say that they are amongst the most powerful drivers of innovation in the field of SQA, both in terms of tools and processes.

With that said, maybe the above question needs to be rephrased: what is it that prevents so many smaller software forges from adopting SQA (properly)?

In obvious answer to that, that these companies lack experience with SQA, is also trite. Experience can be hired.

The real problem appears to be that SQA is not the clearly definable step in the software development life cycle that it appears to be at a glance. You’re all familiar with the waterfall model for software development, and with agile counterparts that iterate over most of the same steps indefinitely.

It’s easy to assume that one of the steps in that famous waterfall model, “verification”, is what SQA is — and that consequently it should happen after (an iteration of) development is done, and that’s the beginning and end of it.

Nothing could be further from the truth.

Properly conducted SQA is not so much verification of your product’s functionality, but verification that your software development process yields high quality at each step.

  1. Requirements need to be verified for their suitability to your business. All too often, product requirements appear “magically” out of thin air, maybe copied from some vaguely related competing product. Those aren’t a bad start, mind you — but for the next iteration of your software, these requirements need to change to adapt to what you’ve learned with the previous iteration. SQA can help in that by making the effects of requirements measurable, and comparing those metrics as requirements are adapted.
  2. Design needs to be verified; at Joost, back in the days when the company was providing a video desktop client, the button for sharing videos with your friends was changed from white to green in one release. Nothing else in the user interface changed. As a consequence, the number of users who used the button increased manifold. Providing metrics “before” and “after” (A/B testing) gives a sound basis for making design decisions.
    But of course this is not limited to visual design alone; in a similar fashion, metrics can verify that a redesigned algorithm increases the performance of your software, for example.
  3. Implementation is the focus of most software testing, and what people usually think of as SQA. The primary focus here is to provide consistently repeatable tests in well-defined environments. Continuous integration provides one of the best methods for ensuring that software works as the developers intended.
  4. Verification encompasses not only testing the implementation, but goes beyond that and includes testing that the software works as designed (whether or not the design is good; that question is answered above).
  5. Maintenance is not a phase usually found in agile development models, as these methodologies tend to loop back to a new requirements phase once software is verified and published. Nonetheless with regards to SQA, a maintenance phase does exist and should exist in the form of regression testing: ensuring that future versions of the software do not break behaviour that was verified in previous versions.

As you can see, SQA touches on all areas of software development. And you can also see, that SQA is largely concerned with providing metrics in order to gain an insight into your product’s performance (in traditional quality management that’s closer to the realm of quality control).

The role of SQA, then, is not merely to ensure your product works. It is to ensure the business you are building around your product works the way you intended, by ensuring that your product’s target audience is satisfied.

It’s worth noting that because SQA and software product development are so closely intertwined, it follows that there is no silver bullet to SQA processes and tools, just like there is no silver bullet to a software development processes and tools. But as with development, there is a rich pool of techniques, technologies and experience to draw on.

And with that said, how can you afford to neglect SQA if you want your business to thrive?

  • Pingback: The Bane of Software Quality Assurance - un|we|sen()

  • mry

    if you talk on quality assurance, you should define term quality as a first thing. Imo it means also things as scalability, reusability, configurability etc. that is why I do not believe in separate qa team that can REALLY quarantee the high quality. Is the term “qa” kind of last season.. at least in agileworld? It is more to train, support and coach chickens in the teams to understand what needs to be done for the quality. It is against agile values if externals will say how teams should work.

  • mry, I think you have a few points here. Let me go through these, not necessarily in the order you raise them.

    1. Think of the the term “QA engineer” (or whatever you call them) as a role rather than a job description. In that sense, any implementor can also be a QA engineer.

    It doesn’t really matter whether QA is performed by a team separate from the team implementing the code, as long as testing is not an afterthought. Agile methodologies try to encourage that by asking developers to write test code first, and then adding implementation that successfully passes these tests.

    This type of test-driven development has downsides as well; the most obvious is that it can lead testers to a certain type of blindness. When you know what your code should do, it’s very easy to write test code that only verifies the known good cases and doesn’t check known bad cases.

    Of course these downsides can be compensated for. One — and by no means the only — method for that is to ensure that the person writing test code for a feature is not the same as the person implementing the feature. Whether that’s two developers in an agile environment, or one test engineer and one developer in a more traditional setting does not really matter.

    2. I could not agree more that scalability, reusability, etc. are also part of “quality”.

    The definition of “quality” that I go by is “it does what it’s meant to do, in (almost) all cases”. If the software is meant to be scalable, then that is part of my definition. If it is meant to be reusable, then that is also part of my definition.

    All of these aspects of quality should — in an ideal world — be part of the product specifications. Assuming such specification exists, “quality” is then achieved if the software meets the specifications.

    Specifications do also exist in an agile world, though usually not so much in the form of a prose document. Specifications in an agile world might exist in the form of test code.

    3. Agile values do not apply everywhere. In automotive software development, where iterating over already-released software is impossible, yet bugs can quite literally kill, you need a different approach to managing quality.

    What agile does, as far as I am concerned, is not so much forbid that externals say how teams should work. Instead agile seems to aim to pull these externals into the team, so they are no longer external. As a result you’ll see far more cross-functional teams in agile environments than in non-agile environments.

Reputation. Meet spriteCloud

Find out today why startups, SMBs, enterprises, brands, digital agencies, e-commerce, and mobile clients turn to spriteCloud to help improve their customer experiences. And their reputation. With complete range of QA services, we provide a full service that includes test planning, functional testing, test automation, performance testing, consultancy, mobile testing, and security testing. We even have a test lab — open to all our clients to use — with a full range of devices and platforms.

Discover how our process can boost your reputation.