In the test automation world the client often requests to run the tests on mobile devices, next to doing the testing on regular desktop. When running your automation in Cucumber, you have multiple possibilities to solve this. In this article I will talk about these possibilities and a new (simple) way to do it with Lapis Lazuli, a Ruby Gem that cooperates with Selenium Webdriver.
I will talk about the following solutions:
- Lapis Lazuli device simulation
- Using Appium
- Using Browserstack
- Using a phone via USB.
Lapis Lazuli device simulation
With the latest release of Lapis Lazuli (version 1.1.0), it is now possible to simulate a mobile device by providing:
- Device name
- Screen width
- Screen height
When starting your cucumber script, all you need to do is add
DEVICE=iphone5. This will start the browser (Chrome or Firefox) with the dimensions and screen size of the defined device.
For more info, see testautomation.info/Lapis_Lazuli:Device_Simulation
- Easy to set up
- Easy to debug your scripts
- Works with all your existing scripts
- Any website will recognise the visitor as mobile.
- Easy to integrate with your existing Continuous Integration
- A free solution
- Not as accurate as a real mobile device.
Another possibility is using Appium. This is a tool that allows you to connect to an emulated device, install or start an application and use it.
- Uses the actual mobile device
- Can also be used in combination with Cucumber
- A free solution
- Does not work with existing TA scripts
- Is more focussed on testing apps, not websites
- Difficult to set up in a Continuous Integration environment
This is a service that allows you to emulate any device. Browserstack also allows tools to connect to it and run automated scripts.
- Reasonably easy to set up
- Close to testing with the real device (emulation)
- Usable in Continuous Integration
- Not a free solution
- Because your scripts are running externally, debugging is difficult
- Remote testing increases the chance of invalid failure due to latency
Using mobile devices via USB
You could also connect a mobile device to your computer and run your scripts on the mobile device using Google Chrome. To be able to do this you will need to install the device drivers from Android Studio
I did not research a solution to do this with iOS, but most likely, it is only possible if you own a OS X computer.
- Most reliable test results
- Easy to do debugging
- All Android tools are free
- A lot of software to install.
- Not useable for Continuous Integration. (Unless you physically connect mobile devices to your CI server)
- Only Chrome is supported.
Chrome uses the same version for mobile and for desktop. Because Chrome has become the most common browser for mobile devices, a simulation like Lapis Lazuli offers, is now quite reliable.
For obvious reasons, Lapis Lazuli calls the mobile web testing “Device Simulation”. It is a solid and simple way to test websites for their mobile capabilities. Unfortunately, it is not the real thing. If using real devices is a priority for you and you want it to fit in your Continuous Integration, I would recommend using Browserstack. If you are only running your tests locally, I would recommend running your tests via USB.
Our solution for Continuous Integration, called Calliope, also supports remote browser testing. This function is still in beta testing, but soon you will be able to link your Browserstack account to your Test Automation solution and run all your scrips on any Browserstack device.
Gijs Paulides – spriteCloud – functional test engineer