The Muse that Helps Your Test Suites Tell Their Stories
Tests answer questions. Test results, over time, tell stories. But, if these results are not shared with stakeholders in a language they can understand, the important lessons these stories tell will not be learned.
Far too many testers and developers have invested their time and your money in building automated test suites with free (like speech not beer) open source software (FOSS) without getting full value for their effort. Having the technical ability to create and run automated test suites on one’s own PC is one thing. Running many different tests against many different environments according to a disciplined plan, consolidating output logs and publishing understandable results to all stakeholders is quite another.
spriteCloud Calliope provides a well-managed and robust framework for executing automated test suites and harnessing the results for professional decision making. Calliope lets your Cucumber test suites tell their full stories. Below a grasp of the feature set:
Scheduling test runs
Calliope provides a scheduling mechanism to enable you to run your tests whenever you want on any interval you like.
Test results that make sense
Calliope uses the cucumber format that makes test results accessible to, and understandable for, non-technical testers and management.
Providing historical analysis of test results to support management go/no-go decisions.
- Providing a stable server stack with mutually compatible FOSS versions
- Managing permutations of test suite subsets and target test environments
- Managing permutations of target OS and browser combinations
- Capturing test outputs and relating them to test suite, target environment, OS and browser combinations
- Analyzing and formatting outputs to enable quick identification of passed and failed test scenarios
By professionalizing test suite execution and reporting, spriteCloud Calliope puts the automated test suite in a central position to:
- Drive effective agile web development approaches like Behaviour Driven Development (BDD)
- Provide timely and comprehensive regression testing across the spectrum of versions and development environments
- Enable robust and reliable continuous integration
- Establish regular monitoring of web app availability
Automate or Stagnate
Only by mastering the disciplined and integrated use of test suite automation can web development teams meet present and future competitive challenges. Understanding how to use test suites as central starting points for all stakeholders unlocks the potential of agile development approaches such as BDD. With automation, comprehensive regression testing becomes feasible; with comprehensive regression testing, continuous integration becomes reliable and robust. Where test suite automation is misunderstood or written off as not being economically justified, the ability to deliver and sustain long-term, high value web products will not develop in pace with market conditions and customer demands.
The Choice for Ruby, Cucumber and BDD
spriteCloud Calliope will execute test suites created with Ruby-based Cucumber and the associated Gherkin structured-English specification language. Test suites can be executed against any user environment based on user-supplied URLs. Execution will take place using Linux, with the user selecting between Firefox, Chrome and Chromium as browsers. The ability to use Selenium Grid to expand cross-browser testing to include Windows, OS/X, IE, Safari and more is on our development roadmap for summer delivery.
spriteCloud – Contributing Member of the Global Cucumber Community
At spriteCloud we love Cucumber. It provides a non-technical structured-English framework in which all stakeholders can agree the required behaviour to be produced by each successive sprint. This becomes a shared starting point for budget holders, users, developers and testers alike. For budget holders it describes business value; for users, acceptance criteria; for developers, requirements; for testers, a test plan. Non-technical testers can work at the structured-English level to develop and add test cases using the wide variety of test case selection methods already in common use. Test automation engineers prepare, in parallel, the code required to navigate websites, locate and use input fields, buttons, etc. and capture and evaluate application responses.
From more than five years of experience of automating Cucumber test suites spriteCloud has developed and released Lapis Lazuli – a Ruby-gem that simplifies the work of building robust, easy to maintain test suites – as a give-back to the FOSS community.
spriteCloud – Integrating Effective Testing into Agile Development
Knowing nothing other than high-tech start-ups and Agile, spriteCloud has created an optimum blend of mature process consultancy and testing services for Creative Agencies and Digital Media production companies, for whom the cost and scope of mainstream quality services and products are often out-of-reach. Need help getting past the “we don’t need/don’t know how to use testers” phase? Contact us at firstname.lastname@example.org.
Do you want your testers to focus on the high-level test cases, but don’t have the in-house skills for the technical automation? For Test Automation Engineering services and Test Automation training contact us at email@example.com.
Want to Make Music with Calliope?
Contact us at firstname.lastname@example.org.