Cucumber and Gherkin:

a guide to open-source automation testing

Quality assurance software testing, and particularly test automation, are our bread and butter here at spriteCloud. Sure, we also provide our clients with cybersecurity penetration testing, manual/functional testing and performance and load testing, but our core area of expertise remains helping clients achieve the benefits of automating their software tests.

To this end, we love open-source tools and we have a particularly strong romance with Cucumber and Gherkin.  As testers we look at the world through the lens of software testing. We often work with clients to help them achieve their business goals through high-quality testing, which involves helping to link testing to development and to business goals. This is at the core of Behaviour Driven Development (BDD), to encourage the collaboration between the business, developers, and tester. We’ll talk about BDD a little more later.

As Cucumber and Gherkin embodies many of the principles we hold dear at spriteCloud – quality, collaboration, and open-source (just to mention a few) – we have written a lot about using Cucumber and Gherkin for software testing. This article is, therefore, intended to serve as our main knowledge resource.

Cucumber and Gherkin inextricably linked to Test Automation. Similar to this guide, we’ve put together a Test Automation guide to help collect our knowledge on the topic. Head to our guide, ‘Test Automation: nearly everything you need to know.’

Share this guide:

Table of Contents

Section 1

Cucumber and Gherkin: Important Concepts

Cucumber and the Gherkin syntax are spriteCloud’s preferred tool and language. This is because they are open-source (making them widely available for diverse teams) and because the Gherkin syntax allows easier readability of test scenarios helping to bridge the gaps between stakeholders. To give more insight into why we love the powerful duo, we will explain a few important concepts.

Section 1.1

What is Cucumber?

Cucumber is an open-source, software tool that supports the Behaviour Driven Development (BDD) framework for writing automated acceptance tests. Cucumber is written in the Gherkin language to define test cases, more of which will be explained later on. Cucumber was originally written in Ruby but now supports various other programming languages like Java, JavaScript, Python and .Net. Cucumber is often used along with Selenium and Watir (amongst others).  Our setup guide will explain how to get started with using our preferred setup of Ruby, Cucumber and Watir.

Section 1.2

Behaviour Driven Development (BDD) with Cucumber

Behavior Driven Development is an agile software development process that is designed to encourage collaboration between the ‘three amigos,’ i.e. business managers, developers, and testers. BDD was born out of test-driven development and is about rethinking the approach to unit testing and acceptance testing. 

According to Wikipedia, the Principles of BDD are:

  • Define a test set for the unit first;
  • Make the tests fail;
  • Then implement the unit;
  • Verify that the implementation of the unit makes the tests succeed.

Domain-specific scripting languages are converted into natural language statements that can express the behavior and expected outcomes as executable tests. Unit tests should be written in order of business value, with acceptance testing being written using the standard agile framework, i.e. a user story using the classes ‘Given’, ‘When’, and ‘Then’.

Read more about Behavior-Driven Development for Cross-Functional Teams with Cucumber in our article. In the article, we provide an example of a test written without Cucumber and then show how that test should be written using BBD and Cucumber’s Gherkin.

Section 1.3

User Story

Behaviour Driven Development specifies that desired behaviours should be written in terms of user stories, which should be documented. While BDD does not have exact requirements for how user stories are to be written it does insist that teams determine a standardised format to be used.

User Stories should generally follow this structure:

Title

An explicit title.

Narrative

A short introductory section with the following structure:

    • As a: the person or role who will benefit from the feature;
    • I want: the feature;
    • So that: the benefit or value of the feature.

Acceptance criteria

A description of each specific scenario of the narrative with the following structure:

    • Given: the initial context at the beginning of the scenario, in one or more clauses;
    • When: the event that triggers the scenario;
    • Then: the expected outcome, in one or more clauses.
Section 1.4

Example of User Stories

Obviously, this is a dumbed down example of a user story to illustration the format of one. In the wild, however, you won’t want to make them too complex or convoluted, as they should be simple and clear. The benefit of doing this, which will be explained in more depth in some of the articles this page links to, is the simple, modular user stories can be reused. Tags such as @ should be used differentiate between different scenarios.

Title: Returns and exchanges go to inventory.


As a store manager,
I want to add items back to inventory when they are returned or exchanged,
So that I can track inventory levels.


@scenario1
Scenario:
Items returned for refund should be added to inventory. Given that a customer previously bought blue suede shoes from me And I have three blue suede shoes in inventory, When they return the blue suede shoes for a refund, Then I should have four blue suede shoes in inventory. @scenario2
Scenario:
Exchanged items should be returned to inventory. Given that a customer previously bought red baseball cap from me And I have two red baseball caps in inventory And three black baseball caps in inventory, When they exchange red baseball cap for a black baseball cap, Then I should have three red baseball caps in inventory And two black baseball caps in inventory.

Now that we’ve explained Behaviour Driven Development and given a brief primer on how to write the user stories that are so central to BBD, let’s look at Gherkin.

Section 1.5

What is Gherkin?

Gherkin is Cucumber’s language parser, which allows software behaviours to be specified in a logical language that people can understand. This means that Cucumber feature documentation is written in business-facing text that is non-technical and human readable for stakeholders like business analysts and managers. This is achieved because Gherkin’s natural language syntax is designed to provide easy to understand documentation of the code under test. Gherkin is written in a syntax that supports BDD.

Section 1.6

How does Gherkin work?

Gherkin works by creating what are called Step Definitions which basically connects the Gherkin steps to code. Gherkin also supports keywords in different languages (like French) and step definitions can now be written in different programming languages (like Javascript).

Below is an explanation of the basic structure of Gerkin but for a more complete explanation and example, visit our article titled ‘Cucumber Basics: Gherkin and Files.’

  • Cucumber tests are divided into ‘Features’, which are use cases that describe a specific function being tested.
  • These ‘Features’ are further divided into ‘Scenarios’, which describe a flow of event and map 1 to 1 with an executable test case for the system. ‘Features’ often have multiple ‘Scenarios’.
  • “Scenarios’ are defined by a series of ‘Steps’ which describe the flow in preconditions, actions, and results in keywords such as; Given, When, Then, And, and But.

This syntax and structure can be seen in the previous example.

Section 2

The Basics of Cucumber and Gherkin

On its surface, writing test scenarios in Cucumber and Gherkin is pretty straight forward. However, as you become more experienced with Gherkin and Behaviour Driven Development, you will start to adopt practices that make your life easier. This section will try to help you shorten that learning curve a little.

  • In an article, our head developer will take you through a step-by-step guide on how to correctly write and execute test scenarios, and help you create tests scenarios that are durable.
  • In another article, we will talk about some of the most common mistakes while writing Gherkin features, like for instance, writing scripts in first person, not third person.
  • We also take a moment to look at how more complex workflows can be described using Cucumber.

With these three articles under your belt, you will be ready to hit the ground running and hopefully avoid some of the common pitfalls that even experienced developers and testers run into.

Section 3

Setup Guides

Now that we understand the key concepts of Cucumber and Gherkin and how they work (more or less) we need the tech stack to put them into practice. As mentioned previously, we love open-source tools and frameworks here at spriteCloud. With that in mind, we’ve created a few guides to help you use Cucumber and Gherkin and several other open-source tools so that you can automate tests for free.

In these three guides, we explain how to set up Ruby, Cucumber, Selenium and Watir, as well as using our specially development Watir helper framework Lapis Lazuli*. Lapis Lazuli provides Cucumber helper functions and scaffolding for easier web test automation suite development, with a lot of the functionality aimed at making Watir more convenient to work with.

Also, as DevOps continuous to increase in adoption and CI/CD becomes more commonplace, as a test automation engineer, you are going to need to know now to perform test automation within a Continuous Integration (CI) setup. Using this open-source tools we’ve written a guide to help you set up and run Cucumber and Selenium on GitLab-CI.

For more information on CI/CD, be sure to skip over to that section in out Test Automation guide.

*Disclaimer: Lapis Lazuli is open-source and free. It is a framework we’ve created to address some frustrations and to streamline some aspects of test suite development. Feel free to use it. Lapis Lazuli on GitHub and its installation notes.

Section 4

Testing with Cucumber

Learning how to set up a tech stack or tool to test is one thing, but actually understanding how to test with Cucumber is another. Fortunately, as test automation experts and Cucumber fanboys (and girls) we’ve taken the time to explain how to use Cucumber for web testing, testing in the cloud, mobile app test automation, and mobile automation with Calabash.

Section 4.1

Web Testing with Cucumber

Website development projects are the most common development projects out there. Due to their pervasiveness and relative low cost, web projects testing does not get as much attention as other project. Arguably, since most websites are fairly simple testing should not be so difficult, but depending on the purpose of a website, a simple bug could be damaging to your organisation.

Such a bug could be as simple as a broken link that causes advertising money to be wasted, or in a more complex example, an eCommerce check-out flow that does not work properly.

Setting up automation tests with Cucumber is a great way to regularly identify defects introduced to the site from both technical and non-technical people. eCommerce sites are more complex and thus require more extensive testing to ensure they are operating properly. For this we’ve written another pose which describes using Cucumber for testing of eCommerce web applications.

Section 4.2

Mobile Test Automation with Cucumber

Mobile applications are taking over and virtually all major players in the digital products and services world has a mobile application to interact with their customers. Whether it be eCommerce with Amazon, music streaming with Spotify or mobile banking, mobile apps are often businesses main method of interaction with customers.

Due to the importance of mobile applications in modern business, sufficient time and effort should be placed in ensuring those apps work as intended. Mobile apps can be tested manually but with Cucumber (and some other tools) mobile app testing can be automated to reduce effort while increasing test coverage.

Learn more about mobile test automation with Cucumber in the posts mentioned below.

Section 5

Cucumber and Gherkin in the Cloud

Being able to create automated tests is just one part of the process of creating amazing products and applications. If your test results aren’t presented to the developers for ops in a useful way, then you spend the time automating for nothing. 

Our test results reporting dashboard, Calliope Pro, makes sharing test results with the rest of the development team easy. Import test results in XML or JSON format manually, using an API call or integrating with your CI/CD solution. Once the reports are run they will automatically get updated to Calliope.

At a glance, the entire development team can see the health of the codebase after the latest changes. Defects are easily spotted and swiftly investigated, making the ‘time to fix’ them very quick.

Calliope Pro works incredibly well with Cucumber but it also supports most of the popular tools and frameworks in the industry. 

Find out for yourself, how Calliope can streamline your quality assurance practices with a free starter account. Give it a try!r