blog

Let's talk about testing

Does “process” have to mean “control” which is seen as “bad”?

Late last week I had the pleasure of visiting a startup that is doing some very interesting things in the social network/online community web space. Like most 2.0 web development shops they consist of a small high performance technical team that work side-by-side a non technical team, each of whom all share equal responsibility for the production of their cloud service. So the developers cut the code that is the running web application, but the ideas that represent the cool innovation in what they are doing – and what gives the company its value – come from everywhere and anybody. As the discussions went on I became intrigued with how they managed all this on-the-fly creativity and how it was fed into the development machinery and eventually turned into production features on their website.

Essentially there was a minimal amount of process as “control” was seen as something that stifles creativity, and slows things down. Given that all the team members sat in the same office there wasn’t a problem with communicating issues so it was easy for them to identify and rectify issues that became problems, and problems that escalated in priority. So it was all good.

Mostly, but it wasn’t all roses and sunshine.

There was some discussions about how they were spending more time now fixing bugs that went into production than there were a few months ago, and the feeling was they didn’t want this trend to continue. In short, they felt the problem was with their QA process, as they all believed that good QA could solve this problem.

Well, yes and no, I thought; a skilled QA person would catch errors during a test cycle, but inevitably testing will only catch 85 – 90 percent of errors in any system. No system is ever error free when it’s put into production. This percentage however is affected by a lot of factors, such as the speed with which a release has to be done, how good the developers are, how conscientious they are with coding standards (which include things like commit reviews, unit testing, adhering to standard conventions), to name just a few. Good development practices always directly influence software quality. Or put another way, even the best QA person can not compensate for sloppy coding from a developer with a poor attitude.

But is there a middle way with process implementation that will support both high innovation and high quality?

I believe there is, and it’s exactly what AGILE tries to do. Remember that AGILE was born out of the need of web developers ten years ago to deliver web services that constantly kept changing from one week to the next. The old traditional methodologies based around waterfall and V model simply didn’t cut the mustard in the new landscape of the web. Instead of trying to control change – and the thing it represented, which was innovation – they embraced it. From that came the evolution of AGILE paradigm and quickly followed by frameworks that supported rapid development.

Really where things go wrong is not when a process isn’t adhered to, because that happens everywhere, but when control is lost. The whole – and arguably only – purpose of a process is to prevent loss of control, and thereby the onset of chaos. While some people I’ve worked with believe that there is such a thing as healthy controlled chaos, I don’t. Any type of chaos means you are not in control of what you’re doing, and at any point in time, inevitably, bad things will happen, and they will impact your ability to deliver work and meet deadlines.

Process really doesn’t have to be restrictive, not at all! It simply has to illustrate clearly how work flows and how problems are handled, both at individual, team and organisational levels. When everyone in a company has exactly the same understanding of who the internal customers are of their work, and how those customers use their work deliverables, then like a water park, everything is going to flow in predictable ways.

Good processes are continually changing as well. The nature of web development is one where what worked today is probably not going to work next month, let alone in six months, but this doesn’t mean your processes should be abandoned; when unpredictable things happen, situations change, or they stop working, then adapt them to something that does work. There isn’t anything wrong with that. In fact it if you’re not continually adjusting your processes they will eventually be ignored, which is worse than not having processes in the first place. Why? Because then it’s all just wasted effort.

In a shop that has adopted them properly, “process” should equal “ubiquitous repeatable success”, which is a very different thing from “stifling control”. Like everything else, it’s the attitude with which it is approached. But that’s something that is outside of the influence of anything other than the individual. Which is why it’s so important to get people with open minds into your teams, and remove those that do not. The quality and success of your software depends on it.

Andy.


  • There are two things people sometimes misunderstand about process, IMO.

    The first one is that a process is seen as restricting freedom. The thing is, a good process shouldn’t really be noticed. It should be, to a large degree, fitted around what your development team likes doing anyway. There’s a danger here in saying that your development team likes not having a process to adhere to, of course. But trying to fit your team into a form that simply isn’t made for them is similarly bad.

    The other thing, which is something of a refinement to this, is that to a degree a process should always be something of a last resort, not the norm. For example, bug reporting should leave a clear “paper” trail, and be detailled enough to properly build test cases from. On the other hand, documenting that something works shouldn’t take any effort at all. I’ve seen processes that treat good and bad cases alike, and impose a lot of work on developers for either. You can imagine how unpopular those were!

    The upshot is that not every process fits every situation and team, and coming up with one that does can take a bit of experimentation. I suspect that involving the team in the process creation can help mitigate some of the most obvious problems.

  • Hi Jens,

    One of the interesting things I’ve found with processes is that you can never get 100% agreement from all participants all of the time; which is separate from actually adhering to a process and executing on work according to its definition. Technical people, especially programmers, are by nature a very opinionated lot who believe they know best, and rarely do you get full consensus between a group larger than five. In nearly all organisations I’ve been in, loose consensus is the model of process of adoption.

    So saying there is no way to actually make a process invisible. The simple fact of not being able to do anything you want, when you want, means it will be obvious to greater or lesser degrees. In which case a lot of process adoption comes down to professionalism.

    Consider the practical example where there is a process in place for making code updates to a production site. In a small team of no more than 5 people or less, changing production directly is not risky, as changes can easily be discussed and agreed upon and made with little to no confusion. However consider a banking environment where there are several hundred developers working at one point in time. Should they all take it upon themselves to change production directly then I’d say within a small amount of time, measured in no more than minutes, that software will stop working. So there is a process by which code moves from development to test and then to production (at its highest level) to prevent this happening.

    A process effectiveness therefore has a directly proportional relationship to the size of a team and complexity of the software system. The larger the number of individuals who are included in a software system, the greater the degree of influence a process has, the greater the reduction in unpredictable results from individual activity.

    Of course no process is ever perfect, and it would be a boring world if there ever was, but there is a definite measurable benefit to process adoption, which is why they exist in the first place, given that these are man made organisational entities. Even in the very open world of open source, one does not simply commit code to the linux kernel – for example – as one feels like it; there is a well structured (arguably very strict) documented approach communicated to developers as process. (http://kernelnewbies.org/UpstreamMerge). Not everyone likes it or wants it, but it’s there for a purpose.

    When looked at this way, process is synonymous with the expression, “for the greater good”.

    Coming back to your discussion points, I would actually disagree with you that a process should be something of a last resort. I would suggest that a process be a first resort so an acceptable implementation can be found that appeals to most individuals, most of the time. That way you encourage adoption and adherence. If you leave process implementation as a last resort, human nature is you won’t like it after being able to do whatever you want. And if you are in a commercial environment where adherence to a process is mandatory, and enforced, you will like it a lot less, and react accordingly.

    Working in non critical new web services companies though does mean that process can be relaxed somewhat as software is non critical when compared to more traditional services such as online banking. If Twitter goes down, it’s annoying; if the Trading software used in the London Stock Exchange fails, financial loss is imminent. Hence, good process could be argued to be less important between these two different types of media. This is from an end consumer point of view. If you are a start-up owner/founder whose company produces poor software which causes consumers to regularly have poor frustrating experiences, then they will ultimately be the ones suffering as finicky web consumers go elsewhere for the same service, and the start-up won’t stay up.

    I do very much agree with you when you say not every process fits every situation and team; like a pair of jeans, one set will fit one group very well, and another group not at all. However, you do not have the option or luxury of wearing nothing at all, so it’s really a case of find the best fit, and then tailor them.

    Good discussion though. Thanks for taking the time to say a few things.

    Andy.

Reputation. Meet spriteCloud

Find out today why startups, SMBs, enterprises, brands, digital agencies, e-commerce, and mobile clients turn to spriteCloud to help improve their customer experiences. And their reputation. With complete range of QA services, we provide a full service that includes test planning, functional testing, test automation, performance testing, consultancy, mobile testing, and security testing. We even have a test lab — open to all our clients to use — with a full range of devices and platforms.

Discover how our process can boost your reputation.