Do I really have to test on all those browser configurations? (Part 1)
One of the most common questions that our clients ask us is whether it is necessary to test on a multitude of browser configurations during cross-browser testing. It’s easy to drown in the choices that are available, making cross-browser testing a difficult process to handle internally.
Since the ultimate goal of test planning is to minimise the risk of high impact, high risk bugs interrupting your website or app’s performance, the easiest place to start is with your product itself and how end users interact with it. Considering the following factors will help you to make an informed decision about which browsers are worth concentrating on during your allocated testing time.
In part 1, I’ll discuss market share and geography and user preference.
The importance of market share
Knowing how your users access your site or app is important as it will help you to optimise your product for success in your target market. Google Analytics is an incredibly useful tool here, as it allows you to see where the largest proportion of your traffic is coming from. It tracks information about your users’ browsers, operating systems, screen resolutions, screen colours, flash versions and Java support, as well as the specific devices your customers use to view your site. Arm yourself with this knowledge, and the browser combinations you choose to test on can be specified to your visitors’ needs.
This sort of data also comes in handy when working with an external testing partner, as they can properly advise you on what combinations it would be useful to test. That way, you get the best test coverage your budget will allow. We advise our clients that testing on combinations with less than 5% market share is of little value if time and money is tight. However, reviewing edge cases is always advised.
Geography and user preference
It’s equally as crucial to consider geography and user preference. Although Google Chrome currently holds the largest global market share (54% in June 2017 according to statcounter.com), users in different geographical regions may prefer different browsers. For example, in the same month, while Chrome dominated in both Russia and Asia, Yandex Browser and UC Browser come in second at 17% and 11% market share respectively. Therefore, if you were targeting these geographical locations, you would need to ensure your site functions equally as well across these particular browsers as your target market is likely to use them regularly. Don’t do this, and you risk decreasing your conversion rate through low test coverage.
Click here to read about the relevance of browser functionality and the differences between devices in part 2.