Skip to content

RPG Developers Should Implement Experience Assurance Testing

In this article, I share why I think we should learn from past mistakes and implement unbiased testing of a game to ensure the quality in a general manner. I call this Experience Assurance Testing.

Photo by Mapbox / Unsplash

Over the years, I have played many role-playing games and experienced many bugs that influenced my opinion of those games. Every time an obvious bug surfaced while playing a game, I would think: this bug could have been found and squashed if someone had just played it before releasing it, as an average player would.

Looking forward to future games in the franchises that I love, I would like to share my thoughts on what is generally off-putting for everybody and what we can learn. I also offer an introduction to Experience Assurance testers (not the same as UX). Let's dive in.

We're Here For The Story

RPG Developers, please don't make your game multiplayer-focused. The rich and diverse worldbuilding and character stories draw players to your RPG universe. Make the lore behind your world extensive and your NPCs an excellent conduit to immersion.

But focusing the game on Multiplayer, while generally more profitable, is not the way to win a gamer's heart. I'm here for the story. Multiplayer can wait until I've satisfied my need to immerse in this universe.

Mass Effect 2 is the perfect example of a story done right. That game's storyline was a frame story that depicts Commander Sheperd collecting an entourage of teammates from around the galaxy to help with a galactic threat. Each of these characters felt like a real person to me. They had their backstories and missions to complete to feel genuinely invested in Sheperd's mission.

Mass Effect 2 is not about the Collectors. It's ultimately about these characters and their micro-stories within the frame story of the game. It's about their interactions with one another and their interactions with Sheperd. Mass Effect 2 took our decisions throughout the game and how our teammates reacted and shaped the ending based on that.

If your teammate doesn't fully commit to your mission, they have a significantly higher risk of dying during the final sequence. The ending of the game would reflect such a death.

All this is to say that we, as players, felt our choices mattered. The story was robust, and our decisions affected the characters around us.

The story is remarkable, but what about the ending, the conclusion to the story we've experienced?

The Ending Should Blow Us Away

As a writer, I know how hard it is to come up with a satisfactory ending.

For example, I played over a hundred hours on Mass Effect 3, doing every little mission and side quest to raise my galactic readiness. And when the time came, all my hard work was poured into a single choice: Red, Blue, or Green.

What about all the missions I did? All the allies I gathered? My teammates? The DLC released after the game's initial launch to amend the ending is a testament to its importance.

The ending should reflect the choices we made during the game. The conclusion and the main storyline cannot feel disconnected from one another.

One reason such disconnections happen in video games is the presence of bugs.

Squash The Bugs

There is one video games company I applaud for squashing their bugs better than anyone else. If you've heard of Cyberpunk 2077, you know I'm talking about CD Projekt Red. They refused to announce a release date because they wanted their game to be bug-free. They did this with Witcher 3: Wild Hunt too. They advanced the release date of the much-anticipated game by six months to work on bugs alone.

Inspiring, no?

I love RPG games, and many gamers will agree when I say that I'm willing to wait for a perfected game instead of a rushed one. The buggy user experience repeats itself in many triple-a games even today. The irony is these developers have more money to perfect the game. They also have corporate goals. The answer to said goals is Day One patch. The developer abides by the corporate requirement to have the game released by a specific date and works on patches as it comes out instead of releasing the game with those fixes.

Day One patch is not the solution.

There will always be bugs, but they should minimally affect the experience for the players and in no way affect the game's overall feel.

As a Software Engineer, I can estimate the QA process for such a game is enormous. And still, someone should come up to management and say, "listen, there are these issues. Can we get some more time to perfect them?" You may receive a no which will cause a crunch. That's what we do when we want to release a quality product in a short amount of time. You may receive a yes, which you'll need to spend wisely.

I'm sure the right people had done that within some of the developer companies releasing the games we all love that had terrible releases. I hope their message is better received next time.

Microtransactions Break Immersion

I only expect to see microtransactions in multiplayer mode. That's the only place where it doesn't feel weird to me to engage with them. The negative effect entering my PayPal credentials in a pop-up inside the game, or worse, a browser window back on my operating system, has on immersion is irrevocable. I immediately lose interest.

Their placement is not the only problem. It's deeper than that. Microtransactions in paid games are frowned upon. I mean, I already paid. Why should my experience be any different than someone who bought something extra through microtransactions? Multiplayer is a different story, but the main single-player game should be untouched by microtransactions.

Games like 'The Witcher 3: Wild Hunt', 'Pillars of Eternity' and its sequel, and 'Divinity: Original Sin' are all very successful role-playing games with no Microtransactions in single-player mode. They prove the game will pay for itself if the story is compelling.

RPG Developers should consider limiting microtransactions to the multiplayer if their corporate goals must include them. Players love and thank developers who heavily invest in their immersion.

Experience Assurance or XA Testing

I don't fancy myself as someone who coins new terms, but I am a writer and haven't seen XA Testing anywhere else. XA is not to be confused with UX, which mainly deals with how users interact with the UI of the game and if it's a good experience or not. XA testers deal with the story, the feel, the immersion, and the overall package. Experience Assurance is so much more than just the UI and interactiveness.

I believe that all game developers should employ XA testers. All of the points I mentioned above would have been identified by Experience Assurance testers easily.

In my ideal vision, XA testers come to the office daily and play the games they're testing as regular players. The emphasis is on playing the game as a whole experience instead of checking whether that mission is working or that system has bugs. These testers do not have much to do during the development stage as there is no game to experience. Frankly, they shouldn't have any technical experience with the game. That can affect their judgment of it.

XA testers can also be just regular players that sign an NDA before the game is released. Many game developers allow players to play a beta version, which does not reveal the story, only a piece of the game.

Instead, XA testers should be able to play a release candidate or two and write a thorough review of what is off-putting in their opinion on all aspects of the game. Some of them will focus on the story itself, others on the graphics, the interactiveness, the characters, the mechanics, anything really. They will have fresh, varied perspectives the developer can learn a lot from.

An even better approach to utilizing XA testers' fresh perspectives is implementing their suggested changes and allowing completely new XA testers to test the next RC. This way, the players do not focus on issues previously reported and find new problems entirely.

A game developer does not need more than two-three rounds of XA testing. Essentially, all the 'outrage' that players would have raised will happen in-house instead of on Twitter and Facebook.

Players will always find something to complain about, but fixing the most pressing issues that XA testers will find instead of customers will dramatically raise the game's reception.

XA Testers Are An Essential Expense

Many developers will probably deny the idea of XA testers since, aside from playing the game and writing a review, they do not assist in the dev stage or writing the story. They are primarily an expense that not all developers would be willing to pay.

But before I go, here's a question for you. What's worse? Spending a couple of months worth of pay for XA testers and, based on their feedback releasing a more robust, user-friendly product — which is more likely to hit those corporate goals, or save that money and lose a whole lot more with awful reception and bugs that influence the overall first impression a player has on the game?

You decide.