Support: How to Collect Data on Bugs & Hand It off to Developers
The fourth article by Ekaterina Rogacheva from Azur Games in collaboration with Ansvery.
In previous articles, we took a close look at why support is important, how to approach support reactively and proactively, and how this will positively affect your product’s user experience.
In this article, we will discuss support as an effective tool for data collection and talk about the best ways to submit the information you’ve collected to the developers.
The faster you get information about new issues and bugs, the sooner you can fix them.
Support can be a very useful tool for collecting data because it’s the first place players go when they run into trouble. In fact, sometimes support can accumulate actionable data before it’s even reflected in analytics. So let’s take a look at the sources from which you can get this info.
1. User requests
Let’s start with direct requests to support. Users come to support for a variety of reasons. They often have questions about game mechanics or suggestions, and, of course, sometimes they run into various issues and bugs. Since our focus in this article is collecting data about bugs, let’s talk about how to improve the process of sorting requests and gathering concomitant information that can be of use to developers when researching an issue.
-
Let users choose a category for their request before they submit it
Assign each category an individual tag that will be added to the request as it’s submitted, then have different kinds of requests automatically sorted into various folders using these tags. This will not only let you keep different types of requests separate, but also establish the extent of a given problem. For example, if your usual request volume for connection issues is 100 emails a day and you suddenly see 200-300 connection-related tickets, you can assume there’s been a server malfunction.
-
Request or automatically collect more detailed information
When a new issue appears, the developers might need additional input in order to resolve it quickly. Make a list of additional kinds of information that will be added to the request. This could include player ID, device name, platform and device OS, player level, server (if you have more than one), and whatever other data you might need. This info can be requested manually, but, first of all, this will add unnecessary steps to your interaction with the user, and second, you will have to spend precious time requesting this information.
2. Store reviews
Replying to reviews lets you not only positively affect your rating and increase players’ loyalty to the company but also collect data on bugs. Some bugs can prevent players from logging into the game in the first place, so your players obviously won’t be able to report these issues from within the game. In this case, players often ask for help through in-store reviews, and it’s very important that you keep an eye on these reviews, since these kinds of problems are critical. The App Store and Google Play Store give you the option to check reviews and reply to them via their admin consoles, but these consoles have a limited set of filters you can use, and the interface isn’t very user-friendly. There are currently a few ready-to-use tools on the market that make review-handling easier. We use AppFollow because it allows us to filter reviews properly and tag them, collect comprehensive analytics, and a lot more. It also lets you sync with Slack – just select the project and the rating you’d like to receive a notification about, and all matching reviews will be sent to a separate chat. This lets you and the development team stay up to date on the latest reviews.
3. In-game polls
One way to collect more specific feedback is to create custom poll windows within the app. These windows will be displayed to users during various stages of their interaction with your game and offer them an opportunity to rate a certain interface element or share their opinion about what they might have liked or disliked. For example, you can show the player a poll right after they finish their next game session. The poll will give them a chance to rate the connection, balance, and whichever other aspects you’re interested in. In the article about approaching your target audience I mentioned the “rate us” feature that also involves integrating an in-game poll. Think in advance about how to download the data you’ve collected in order to process it as quickly as possible.
Communication with the development team
So, you’ve processed user feedback and collected data on new bugs. Now your goal is to hand this information off to the developers and gauge the extent of the issues. Usually, only a small percentage of users report a problem to support, so you need to realize that for every person who contacts support there are many more who don’t, but still experience the same issues. In order to let the developers evaluate how serious a problem is, follow these tips when generating reports:
-
For every individual problem, state its percentage in the incoming ticket volume, not the specific number of complaints – this will be more indicative of the project.
-
Take a look at how the number of complaints you’ve received in various categories has changed over time. This way you’ll notice if an issue has suddenly become critical.
Keeping an eye on the varying percentage of complaints in given category overtime isn’t always enough. For example, let’s say that over the past month 1,000 people have reported issues, and 10% of them, i.e. 100 people, were complaining about a bad connection. Then, after the next update, you get 200 reports, 20% of which, i.e. 40 complaints, turn out to be connection-related. This means that if the number of installs remained the same, the situation in regard to connection quality has actually gotten better despite the fact that the percentage of complaints has increased. This is why you need to pay attention to categories where the percentage of complaints stays the same and compare the remaining categories to them. You can use categories such as complaints about forced ads, high prices, pay-to-win, etc. as reference points.
-
Generate a report for each version of the app in order to see how the number of reports in each category has changed over time.
When generating a report, be sure to add additional information you’ve collected from players (whether automatically or manually). As I mentioned above, this information could include player ID, device platform and OS, player level, server, and any other data that might be useful.
In conclusion, I’d like to point out that collecting data about issues and delivering it to the dev team is one of the most important functions of support. Moreover, it’s relevant for both indie developers and big companies. This is why you need to pay close attention to the optimization of the data collection process during the initial project launch phase, as it can directly affect the quality of the product and help you develop and release any necessary fixes as quickly as possible.