With a career that now spans an incredible five decades, Julian 'Jaz' Rignall is someone who, for an entire generation of gamers, will need little in the way of introduction – but we'll do one anyway because his career has been truly remarkable.
A gamer from a very early age, Rignall beat 500 other hopefuls to win the CVG championships in 1983, after which he joined publisher Newsfield and became part of the legendary team behind Zzap!64, one of the UK's most influential video game magazines. He later joined EMAP and quickly became editor of CVG, transforming the publication's fortunes as the gaming world shifted from home computers to Japanese consoles.
While at EMAP, Rignall's guiding hand helped steer the company towards bumper circulations and astonishing mass-market appeal; he helped launch Mean Machines – a console-focused mag which would arguably lay down the template for others to emulate – as well as Megatech, Nintendo Magazine System and Mean Machines Sega.
He then crossed over to the other side of the fence, joining Virgin Interactive, where he worked on titles such as Agile Warrior F-111X, NanoTek Warrior, The Lion King and The Jungle Book. In 1997, he moved back into games journalism by helping to establish the Imagine Games Network, which would later become the behemoth that is IGN. Rignall's tenure as Editorial Director helped shape the voice of the site in those early years.
After stints at retail giant Walmart and the Bank of America, he joined Future Publishing to help launch its Future Plus custom publishing division, and then, in 2010, became vice-president of content at GamePro Media, publisher of the legendary video game magazine of the same name. When GamePro folded, he joined the team at US Gamer, acting as Editorial Lead and Editor-at-Large during his time there.
Today, Rignall is putting his considerable experience to good use at VGM, a market research firm which is the largest provider of custom research to the video game industry and has a client list that includes Square Enix, Sony, Sega, Striking Distance, Nexon, Bandai Namco, 2K, Zynga, EA, Activision Blizzard and Tencent.
Keen to learn a little more about how Rignall and his colleagues are working tirelessly to improve the video games we play, we sat down for a chat with the great man himself.
Time Extension: Could you give us a brief outline of the kind of work you undertake at VGM?
Julian Rignall: First of all, I want to make clear that while I often say that I test games for a living, this is neither a review process nor bug testing. Both of those things are wholly different processes. Especially bug, or QA testing, where game testers are specifically looking for technical faults or other issues that might break or cause a game to malfunction.
What VGM – the company I work at – does is game research, and I’m a video game research analyst. My job is to essentially garner feedback from active gamers – members of the general public – and use it to help companies finesse their video games into the best possible products and hopefully realize their full potential.
And that is a surprisingly detailed and involved process.
When a video game publisher decides to test a game or game concept, VGM’s business and project management team will listen to what it is they want to test, and then decide the best course of action.
My job is to essentially garner feedback from active gamers – members of the general public – and use it to help companies finesse their video games into the best possible products and hopefully realize their full potential
Once we understand what it is we’re actually testing and the game’s target audience, our recruitment department will then start calling members of the public – people who are on our books because they signed up to be a tester – who best match that target audience. While they’re on the phone with these people, the recruitment folks will ask them about their gaming experiences to make sure they have the right kind of knowledge we’re looking for, and will often also ask about their gaming tastes. After all, we don’t want to test a AAA RPG using people who don’t like that kind of game. The team then whittles down the list to a group of potential testers who are not only right for the test, but who also come across as good potential candidates because they are knowledgeable and articulate.
While that is happening, the qualitative and quantitative teams also spring into action. The former – people like me – specialize in verbal and written focus group feedback and writing the bulk of the analysis, while the latter are statistical specialists who do all the data analysis. Things like looking at the surveys that players fill out to rate features such as the graphics and sound, or stuff like how fun the game is, or how appealing its characters or story are.
We do two things at this juncture. The first is to create a playtest survey asking for players to rate different aspects of the game, often with follow-up “open-ended” questions that ask them things like why they rated something the way they did, or requesting more details as to why they like or don’t like a specific aspect of the game. And the second is creating a focus group discussion guide that contains loads of deep-dive questions that we use during the focus group discussions that happen after our test participants have played the game. These comprise a smaller group of people who we feel are the most knowledgeable and articulate candidates, and basically, our moderator talks to them to garner lots of very specific info. The focus groups are an extremely important aspect of the test since this is where we can really drill down into specific topics, and our moderators can ask questions so that we can really understand what players do and don’t like and why.
Once these are created, we then go back and forth with the company whose games are being tested to make sure we’re covering everything that they want us to test, and then we arrange a period of testing, which is sometimes a few groups over a single day, and sometimes many groups over several days, depending on what kind of test it is. And often, the different groups are different kinds of gamers. Perhaps people who have tons of experience playing a game like the one being tested, along with those who like the idea of it, but have little experience with it. All those different perspectives can really add depth and detail to our findings.
In the meantime, our facilities team will work with the client to ensure that we have the right hardware and the correct build of the game so that everything will run smoothly at our testing facility when we finally conduct the test. We have two of these in the US – located in San Francisco and New York – and one in Brighton in the UK.
On the day of the test, the respondents arrived at the test facility at their designated time, sign an NDA (which is massively important), and will then start testing the game. We usually have several of the systems they’re using – either phones, PCs, or consoles, depending on the kind of game we’re testing – hooked up to capture systems that continually live stream what they’re doing so analysts like me, and indeed the client themselves (be they business or marking people, designers or developers) can actually watch them play and see what they’re doing.
As the test continues, the facilities folks constantly walk around looking over peoples’ shoulders to make sure they’re not getting stuck or to give them a bit of guidance if they don’t know what to do. They also help resolve technical issues (which happen quite a bit when you’re testing early pre-release software) and generally keep the test running smoothly.
This is where I use my in-depth gaming knowledge and experience to explain to the client what they need to do to improve their game and help make it more fun and, ultimately, more salable
Once the playtest is completed, we’ll then conduct a round table focus group discussion, usually with four to six people. We stream this in real-time using cameras and mics, and I – and often the client – will watch this. I basically take very detailed notes and observations about not only what people are saying, but how they’re saying it. Because sometimes, I’ll want to stress to the client that not only do the players, say, not like something – but they’re pissed off and angry about it. Or they don’t just like something, they are really excited and animated about it. Those kinds of observations, combined with what they are saying, enable us to give incredibly detailed feedback about specific aspects of the game being tested.
When the test is complete, all the survey data is analyzed by the quant team, and they produce version one of the report that details all the numerical feedback. The qual analyst – someone like me – then goes through all the respondents’ open-ended survey responses, their focus group feedback, and any other relevant info and turns that into detailed feedback that explains what the playtesters like/don’t or understand/don’t understand about the game and why.
After all that is compiled, I then write the final recommendations – the most critical part of the test. This is where I use my in-depth gaming knowledge and experience to explain to the client what they need to do to improve their game and help make it more fun and, ultimately, more salable. Sometimes our recommendations will also involve the game’s marketing, messaging, or even its price point if we receive feedback that we feel requires these kinds of issues to be addressed.
More importantly, we have to make sure our recommendations are also feasible, essentially striking the right balance between effective recommendations that are also doable. That can be very challenging, but it’s my favourite part of the job for sure – combining a weird mishmash of the experience I’ve garnered over the years reviewing, marketing, and even making games – as well as drawing on my overall understanding of the games market and gamers’ tastes in general.
And when all that is done, the report is reviewed by everyone involved to make sure it’s on the money, and once everything is squared away, it’s presented to the client.
Time Extension: How does your work help publishers and developers to shape the games that are eventually released at retail?
Julian Rignall: In all sorts of ways. We do a vast number of different types of tests. Firstly, we test everything from mobile releases to literally the biggest and most successful AAA franchises on the planet. We test game concepts – sometimes literally descriptions of a game, concept art, and perhaps early gameplay videos – to make sure that players actually find these concepts interesting and fun (and if not, we’ll explain what needs to be done to make them so).
We look at games in various states of production to find out what players like and don’t like about them. User experience and general game comprehension tests are also very important. Stuff like making sure the user interface is intuitive and easy to use, finding out whether or not a game’s tutorials effectively explain how to play it and how its features work (or indeed whether or not a game might need more of them if players keep getting stuck or can’t figure out what to do).
We also test some games with parents and kids – family games that might have features such as couch co-op – to ensure they strike the right balance of challenge and fun for players of all ages. Sometimes we’ll test new game features, such as changes to a game’s interface or new modes, new game content updates, or ask what players think about a potential new feature – such as a battle pass – being added to a game.
Sega did recently publicly recognize that VGM helped test Sonic Frontiers – and even put the testing team in the game’s credits, which included me... we definitely helped finesse the game and help knock off some of its rough edges
And sometimes we’re tasked with figuring out what factors are making people leave a massively multiplayer online game, for example, or how much players struggle when they return to a big live service game they haven’t played it for a while.
All the feedback from these tests can really help shape a game’s development.
Time Extension: Do you have any examples you can run us through of where your input has resulted in a vastly different or improved experience in the final product?
Julian Rignall: The problem is that almost all the work we do is under strict NDA, so I can’t really talk specifically about how our feedback might have changed the many games we’ve tested – even after they’re released. However, Sega did recently publicly recognize that VGM helped test Sonic Frontiers – and even put the testing team in the game’s credits, which included me. I can’t tell you specifically about what we did and what changes were made based on our feedback, but we definitely helped finesse the game and help knock off some of its rough edges.
Time Extension: Does your experience as a player, journalist and developer with decades of knowledge help you do your job at VGM?
Julian Rignall: Without a doubt, the experience I’ve garnered not only reviewing games, but making and marketing them – and constantly talking to gamers to understand their tastes and the market in general – really helps me make informed recommendations. However, the cool thing about testing is that it involves different types of people from lots of different types of disciplines, many of which I think members of the general public aren’t necessarily aware of.
Most of the time, when people think about video game jobs, they think about direct video game involvement, such as being a programmer, designer, or artist. Or perhaps they might think about marketing. But there’s tons of other jobs too. I mean, my company employs people with excellent general tech knowledge to ensure the testing facilities work and our tests run smoothly, we have admin people who book people in and out of tests, explain NDAs to our respondents and make sure they’re signed and recorded. We have outreach people who talk to and essentially screen potential test candidates on the phone to ensure we have the right people at our tests. There are project managers and coordinators who talk to the client and ensure our tests are focused on the right topics. And there are moderators who conduct the focus group discussions.
Basically, lots of different disciplines requiring all sorts of different skill sets – that ultimately show that there are now a huge variety of different routes into the video games business.
Time Extension: The games you reviewed back in the '80s and '90s will have presumably undergone relatively little focus testing, market research or analysis when compared to modern AAA titles. How do you feel the market has shifted in that time?
Julian Rignall: Yeah. I’ve been in the industry long enough to see it go from literally like one dude in his back bedroom programming a game on his computer and releasing it via mail order without any kind of testing at all, though the early '90s when “testing” was really bug testing with a bit of bonus gameplay/user experience feedback from the testers (which was often ignored by programmers and developers who thought they knew best), to an industry that now creates products that suck up hundreds of millions of dollars in investment. That is an unbelievably huge shift. There are very few industries that have undergone that kind of change in that amount of time.
I’ve been in the industry long enough to see it go from literally like one dude in his back bedroom programming a game on his computer and releasing it via mail order
Time Extension: Do you think companies such as VGM will become even more important as time goes on?
Julian Rignall: Well, I feel research has been a hugely important aspect of the industry for a good couple of decades now. It just doesn’t get talked about much. I mean, when you think of the hundreds of millions of dollars being invested in games these days, making sure that players will find their subject matter interesting and appealing, understand how to actually play them, and have an enjoyable and compelling user experience while doing so – as well as ensuring that these games offer the right level of challenge so that they are fun to play is critically important. Some of the really big publishers have their own research team that do all this, but most don’t – and that’s where companies like VGM come in.
Time Extension: If someone wanted to get involved in playtests, how would they go about doing that?
Julian Rignall: VGM is always on the lookout for new testers, and since we’ve recently set up a testing facility in Brighton, we’re actively recruiting for potential playtesters based in the UK. If you’re interested in being involved in game playtesting, we'd love to hear from you.
Something interesting to consider is that if you are indeed the right kind of person for one of our playtests, you’ll not only get to come into the office and play something months – and potentially years – before it’s released and get the chance to potentially shape its development, you’ll also earn yourself a decent stipend for doing so, since we don’t believe that any of this stuff should be done for free.