Mobile test automation for 600,000 Ben users

An app that carries the whole customer relationship

Ben is a Dutch budget mobile brand operated by T-Mobile Netherlands, with roughly 600,000 users. The brand keeps prices low through self-service, which makes the app the centre of the customer relationship: 60 to 70 percent of users manage their data bundles, subscriptions and invoices through it.

The team behind that app was small. Two developers from development partner Oberon, plus a product owner. When their shared manual tester left, the scrum master picked up testing on the side.

Manual testing stopped keeping up

User numbers grew, and incoming errors grew with them. Ben initially looked for a single manual tester, but the product owner drew a sharper conclusion: with this many users and this error rate, manual testing alone was not enough. The team needed dedicated testers and an automation suite for the native mobile app, without disrupting a two-week sprint rhythm run across three organisations.

First, a foundation: manual coverage and a regression map

spriteCloud started where the gap was. A mobile test engineer joined the team for manual testing of in-development features and built the project's first complete regression test document, a full map of existing functionality to validate every release against.

The integration went quickly. One embedded tester, working in the team's sprints, using the team's tools.

A business case with a built-in handover

As the app grew, spriteCloud presented a business case for mobile test automation with a clear choice: we build and maintain the suite, or we build it and hand it over. Ben chose a hybrid. Our specialist would set up the framework, our tester in the team would expand it, and ownership would transfer to Ben once it was stable.

That handover was designed in from day one, not bolted on at contract end.

Detox, Cucumber and a shared dashboard, in six weeks

The setup took six weeks. Our senior test automation specialist chose Detox over Appium for two reasons: the Ben app is built in React Native, so the developers could work on tests in the technology they already knew, and Detox tests are less flaky. Scenarios were written in Gherkin with Cucumber, readable by everyone on the team, covering the flows that matter most: the subscription dashboard, buying extra data, and changing subscriptions.

Tests ran in TeamCity on every build, with results reported in Calliope Pro. For a team spread across three organisations, that dashboard became the shared source of truth on app stability.

A handover that actually handed over

The suite was deliberately built to be picked up by new testers: high maintainability, clean structure, full documentation. When spriteCloud left the project, Ben's own team took over the framework without problems and kept expanding it for the most important customer journeys in the app.

Releases stopped being the risky moment

• Six months of releases after the setup went out with no or minimal issues.
• Bugs surfaced earlier, before release instead of after, per the product owner.
• Developers write tests themselves in the stack they already know.
• The suite survived the handover and is run by Ben's own team.
• The app proved stable enough that the team was asked to bring parts of it to the web interface.