Over the last couple of years I have worked on many projects using the Gitflow workflow. This well established workflow works really well for teams working according to Scrum. For a detailed explanation of the Gitflow workflow I highly recommend reading Vincent Driessen’s blog. I would also recommend reading our post about how to setup and run Cucumber and Selenium web tests on GitLab CI. Also, while we are talking about workflow, have a read of our post explaining how best to work with complex workflows with Cucumber. In my blog, I am going to assume that the reader has a prior knowledge about the model described in the post.
Source: Vincent Driessen
When should you start testing
A lot of time during the projects that I have been involved in, I was the first tester in the team. I know: in a Scrum team everybody is supposed to be able to do all tasks and we are all developers. But as I have written in an earlier blog, in the end, you will see that within a Scrum team, all members have specific expertise. You will have your back-enders, front-enders, testers and sometimes designers. When I was the first tester to be added to a team, there was always a discussion about when and where the tester should do his or her testing. The testing situation ‘before’ usually involved developers doing some of the testing locally, then having the Product Owner doing all the testing in a test or UAT environment.
So when I enter a new team, the discussion starts over where to add testing to the mix. Since the team is already working according to Scrum, at the end of the sprint the team should be delivering a potentially releasable product. Assuming the definition of ‘done’ also requires testing to be carried out, you cannot wait to test until the release branch is created. The release branch is usually created near the end of the sprint. According to the Gitflow workflow model, only minor bugfixes and small changes – dotting i’s, crossing t’s – should be done on the release branch. There should be no critical or blocking issues in the release branch. So waiting with testing until the release branch is ready is not an option. It will also put a lot of pressure on the tester and basically make the sprint into a small waterfall, making the whole incremental development philosophy behind Scrum obsolete. And when critical bugs are discovered, you run a high risk of not being able to reach your sprint goal.
1. Create the new release branch at the beginning of the sprint.
A solution to this issue that sometimes comes up is to create a new release branch at the start of the sprint. Finished features are being committed to both develop and release, so the tester can start testing on the release branch from the start of the sprint. This sounds like a good idea, but it adds another layer of complexity in keeping your branches up-to-date.
You can still run into the same issue as when you create the release branch at the beginning of the sprint. It is still possible that the feature branch which is added to the release branch at the very end of the sprint creates a new blocking issue. This can jeopardise the goal of the sprint; having a potentially releasable product ready.
And you can still run into the issue of the last feature being added to the release branch, creating a critical issue which cannot be fixed before the end of the sprint. This means that you will not have a potentially releasable product ready at the end of the sprint. But, depending on the constraints of the development environment, this might be a (far from perfect) way of working.
2. Test the develop branch after new features are merged with the current release.
The next solution offered, comparable to the first one, is to test on the develop branch after new feature branches are merged in Develop. This sounds appealing; you seem to test each new feature separately. But most teams, consisting of an average of 7 developers, will produce new features more often then you can keep up with testing. So you will identify an issue, but since you are testing the work of more than one developer, it can be difficult to identify which new code exactly created the bug. This will lead to extra work for developers to identify exactly where to fix the bug. Unless you use the test results dashboard, Calliope Pro, we describe below.
Still, depending on the constraints of the development environment, this might be a valid way of working. You are working in an iterative way, enabling early feedback to the developers. But testing development might also lead to developers having to spend extra time on finding what is causing the issue.
3. Start testing feature branches at the beginning.
The last, and to me, most preferable moment to start testing is right at the beginning by testing feature branches. You will know exactly which feature and which tickets are there, greatly reducing the time needed to figure out what changes caused the bug. Of course, the development environment needs to accommodate this. In one of my projects, I had multiple test environments available and I was able to build and deploy whatever branch I wanted using a tool like Atlassian Bamboo.
Source: Atlassian’s confluence
What made this flow so streamlined was the fact that a developer and I worked in sync: he notified me when he was ready, I tested his branch, he fixed his issues, I retested the branch, and then we were done. Tested feature branches were merged into the develop branch. At the end of the sprint, a release branch was created on which we only needed to run our test automation set, do some regression testing and maybe fix some minor bugs. But that was it.
The disparity between the number of developers to testers, combined with the fact that the entire code base must be tested for each new feature, means that efficiency is critical for success. Automating testing is one (very good) answer to this issue. Test automation is also suited to testing in various branches on ones CI/CD solution. If you are new to automation be sure to check out guide on test automation.
As we as testers always say: start testing as early in the development process as possible. Start with reviewing documents and test code whenever it is available. Working in a Scrum team in that way will greatly enhance your velocity, which prevents critical bugs making it to production. In the end, leading to a happy customer. If you found this interesting, have a look at our post about Cucumber testing in cross-functional teams with Behaviour Driven Development (BDD).
Working effectively within a scrum team also means being able to track progress while sharing issues and test results with ease. From a test engineer perspective, there is relatively little out there to assist in this area. Until now, that is!
Calliope Pro is a software test results dashboard, independent of any test solution that unifies your results to share with your team. It supports all major test formats and lets you securely store your test results in one location. This provides you with;
- an overview of test result over time;
- the ease sharing results for effective collaboration;
- the ability to incorporate screenshots into test results;
- the scheduling and running of your test on demand.
Try Calliope Pro free today and evaluate if it is right for your team.
Black Friday and Cyber Monday have just passed, although the
Around the time at which Amazon assumed its position at
DevOps! Great! Push it live, quick and fast! DevOps, which