blog

Let's talk about testing

Introduction to game and mobile game testing

This entry is part 1 of 3 in the series Game Testing

In the past decade, games have grown to become one of the largest sectors within the software development industry. With its humble beginnings in the late 1970’s, the games market is now worth around a 100 Billion USD worldwide (figure 1), based on a report from games market research group NewZoo [1] with projections to 2017 hitting $107 billion. In comparison, according to data from market research firm, IBISWorld [2]. Movie earnings are estimated to $90 billion in 2014, with the sector seeing further decline to a projected $89 billion in 2015.

Next to the big growth. the gaming sector is also one of the fastest changing within software development. Looking back over the past 8 years or so, the way gamers “consume” content has drastically changed and will continue to change at a rapid pace.
Firstly, leisure gamers today care less about professionally created content, and instead enjoy watching each other with an increasing desire to create, share and be a part of the experience. The growth of eSports [3], Twitch [4], and the sharing of video game content illustrates this point. Due to this trend, games with the most potential are competitive or creative in nature. This is why creative games like Minecraft [5] have thrived, as well as an array of competitive titles focussed towards eSports.
But next to leisure there’s also a big focus on educational and medical games. According to a study done by the NPD group [6], in 2011, 91% of US children (age 11 upto 17) play computer and video games. And next to this, gamification is used as a great trigger for kids to learn. According a recent study from onlinecollegecourses.com, 70% of the teachers saw an increase in student engagement when using educational video games.
All and all, the games market has become huge and now contains many sub-segments.

Newzoo_Global_Games_Market_2015_Per_Screen_Segment_V1_Transparent1

Figure 1. Global games market will grow 9.4% to $91.5B in 2015, Newzoo, 22 April 2015

Differences between software and game testing

Game testing borrows most of it’s structure from general software testing, but with some tweaks. If you look at a piece of software, it is most likely made to give you as much access to the functions of the program as possible, while keeping the learning curve for the user at a minimum (with some exceptions). With games it is completely the opposite case. As much information as possible will be hidden for the user so the main focus of the player can be on immersion, i.e. ‘jumping into the game’. A game will try to outwit you where possible, adding artificial intelligence (AI) or random actions. It will actively try to keep you from knowing what it will do next. This starts with games as ’simple’ as Candy Crush [7] or Balloony Land [8] to complex games as World of Warcraft [9] or Call of Duty [10].

Looking at 3D games, the combination of AI and random actions makes reproducing found defects difficult. A serious bug encountered by a tester might not be reproducible during the first 100 attempts to recreate it.
Depending on the complexity of the game, reproducing of defects is mostly done with the help of save games when playing offline. Saved games are pieces of digitally stored information about the progress of a player in a video game and allow the tester to attempt to recreate the precise moment a failure occurs by isolating small incremental series of movements as saved games, until the failure is found again.

As an example a tester will save a game every 15 minutes until the defect is found. Loading the saved game, the tester will then save the game every 5 minutes until the defect is found until the defect is isolated.

There are many possibilities to automate testing for gaming. A simple example is to have an AI player randomly walk around in a level to see if it gets stuck somewhere. Another is image verification, using screenshots to verify an outcome with the expected outcome created earlier.
With multi-player gaming, there are many more factors to take into account.

For now, much testing continues to rely on manual methods. When you compare it with conventional software, repeating the same actions normally continues to yield the same result, making it relatively easy for a tester to reproduce a found bug (press a button -> expected outcome).

Next to the 3D titles, there is also a big market appearing on mobile platforms. Since the hardware inside mobile devices is not as capable as that found inside its desktop cousin, the actions that can be performed by the user are limited, the 3D aspect is normally less and there is a bigger focus on social and connectivity. Simply put, most mobile games today are simple or repetitive, designed to play for short periods of time. Here, the gamification concept comes into effect, with a big focus on status and achievement. The idea of beating your online friends by getting a higher score, or passing them by reaching a higher level are the main goals. There are usually more iterations / releases of mobile titles than the big desktop 3D titles, making efficient and effective regression testing even more important.

[To be continued]

References:

[1] Newzoo
[2] businesstech research
[3] eSports
[4] Twitch
[5] Minecraft
[6] NPD Group
[7] Candy Crush
[8] Balloony Land
[9] World of Warcraft
[10] Call of Duty


Comments are closed.

Reputation. Meet spriteCloud

Find out today why startups, SMBs, enterprises, brands, digital agencies, e-commerce, and mobile clients turn to spriteCloud to help improve their customer experiences. And their reputation. With complete range of QA services, we provide a full service that includes test planning, functional testing, test automation, performance testing, consultancy, mobile testing, and security testing. We even have a test lab — open to all our clients to use — with a full range of devices and platforms.

Discover how our process can boost your reputation.