The difference between manual testing and test automation.

While the differences between these two forms of testing is pretty obvious by their names, manual and automated. While it seems clear, it is important to give a fuller explanation to these two different yet inexorably connected forms of tests. If you have spent as much time reading, working, and thinking about test automation as we have, you will begin to notice that not everyone has the correct assumptions regarding test automation and the people that manage those TA processes.


According to Wikipedia:

Manual testing is the process of manually testing software for defects. It requires a tester to play the role of an end user whereby they use most of the application’s features to ensure correct behaviour. To guarantee completeness of testing, the tester often follows a written test plan that leads them through a set of important test cases.

What manual testing can feel like.

Test automation is the use of software separate from the software being tested to control the execution of tests and the comparison of actual outcomes with predicted outcomes. Test automation can automate some repetitive but necessary tasks in a formalized testing process already in place, or perform additional testing that would be difficult to do manually. Test automation is critical for continuous delivery and continuous testing.

What test automation can feel like.

Manual testing

Manual testing is just that, the testing of software by a technical test engineer, by hand. Manual testing is undertaken to discover bugs in an application or system under test (AUT or SUT). Technical testers document the test cases for repeatability purposes and after each test, they record the results of the test. Due to the current methodologies of software development (Agile or DevOps) testing or quality assurance can quickly become a bottleneck in the development life cycle.

This is because as the application develops through the addition of new features and becomes increasingly more complex, so do the tests required to ensure that the actual results meet the expected results of the application for customers and business owners. The reason from this complexity it has to do with the fact while the developers mainly need to focus on creating new code, testers must check that new code and old code continue to play well together. This can quickly add up as seen in the image below. The developer in grey only focuses on new code, while the tester in blue must focus on ensuring coverage for all code (including from previous sprints).

Showing the extra workload testers face at each sprint

Test automation

Test automation involves the use of automation tools to execute test scripts. These test scripts are developed and maintained by testers. They are used to automate tests that are repetitive and time-consuming for testers and are mainly focused on discovering regressions caused by new code. Manual testing identifies the method/process of how a function is to be tested. For example, testing whether the payment process in a check out sequence works. A script is then written to replicate that process to make it possible to be run automatically.

These automatic tests run faster and run during off-hours making the entire testing process much more efficient. As we discussed previously, this need for efficiency is crucial when the workload on testers accumulates with every sprint. In a simple sense, you can say that test automation is a sped-up version of manual testing and with that speed comes the ability to generate lots of data which provide a more reliable understanding of the stability of the system.

A technical tester can test an API a few times in an hour and get an error. A test automation engineer can test the API dozens or hundreds of times in an hour to provide a more accurate picture of the errors and potential causes. When things are sped up the differences become more pronounced, and thus easier to differentiate.

Test automation is critical for continuous delivery and continuous testing.

In modern agile software development methodologies, test automation is a must to meet budgetary and release targets. Proposing to a product owner that after every sprint you must hire another two testers would probably make them reconsider your employment.

The Continuous Integration and Continuous Delivery (CI/CD) loop

Test automation can not replace manual testing completely

While test automation clearly has its benefits, it is not a replacement for manual testing. It is a tool for supporting and increasing the efficiency of testing as a whole. Test automation is designed to take easily repeatable tests and perform them much faster for the tester.

Who do you think writes and maintains the test scripts that are to be automated? For that matter, who would be the person to perform exploratory testing in the first place to understand the areas of the application that require test coverage? That’s right, a technical tester performing manual testing. A manual tester’s skill and knowledge are required to identify tests to be conducted, perform initial manual tester, evaluate the need for repeated testing, and to ultimately produce and maintain the test scripts which are automatically run. Scale this up to dozens or hundreds of independent tests scripts, and you have a test automation suite that allows you to monitor your software development project rather effectively. A testers time is then typically split between developing and maintaining test scripts (maintenance), 80%, and performing manual (exploratory) testing, 20%.

While completely automating testing roles is not likely to happen in our careers, we are beginning to see more tools that provide record and play functions and AI/machine learning assistance for test automation. It would take an extremely naïve, or perhaps incredibly innovative, product owner to pass the reins of test over to a computer completely. After all, who will test the tester?

A GIF of the reporting tool

Calliope is a test results dashboard which allows developers, product owners, and software testers to collaborate more effectively. Calliope Calliope serves as one location to share, compare, and monitor test results from all of the testing tools you use, and therefore, track the health of the development project. At the moment Calliope only fully supports Test Automation, but soon it will support manual testing results, security testing results, and link checking results making it the DevOps tool of choice.

Give a try today with our free starter account.

Suggested Posts