A bugs life[Quality assurence]

[Ace Pace]

You know, I havn't been in this situation,but having worked abit in QA, this is kinda representative.

Is it still worth hunting bugs for a living - at the risk of being treated like an insect - for your break in the videogame industry?

“You are not playing games.” While researching the life and times of the modern-day games tester for this article, we asked everyone we interviewed – quality assurance professionals, the developers who work with them (and as often as not used to be them), disillusioned former testers who’ve hung up their gamepads and walked away – what the most common misconception about the job was. And without exception, they all gave that same answer. Anyone with any experience of the QA process will deny the slightest resemblance between testing a game and playing one for pleasure: finding bugs is unmistakably work, and, by common consensus, very dull and repetitive work at that. On top of this, pay is often poor, job security frail, working conditions extreme and recognition hard to come by.


So why do it? Most of the thousands of hopefuls who constantly shower publishers and developers with CVs share a passion for gaming that impels them to try to get closer to its source. Some of those are under the mistaken impression they’ll be being paid for playtime. A precious few will harbour a genuine taste for the work, and an ambition to make a career of it. The majority, though, will be hoping that breaking games for a living will provide a back door route to making them – either because they have no relevant qualifications, aptitude or experience, or because they do have those things but, despite that, the front door still remains steadfastly, frustratingly shut. It’s worked for a great many in the past, but with the changing standards and methodologies of a maturing industry, can it still? And what will you have to go through en route?

Getting through that back door in the first place is far from straightforward. Competition is intense, and there are practical hurdles too: QA departments’ workload is very variable, so they usually only offer short-term contracts – three months is normal – and require immediate availability. But many employers put more emphasis on personality and passion for gaming than the usual criteria of education and experience. “Whatever A-levels they’ve got is irrelevant,” says Andy Robson, head of testing at Lionhead, which has year-long queues for its global schools work experience programme. “It’s really not about qualifications at all. It’s enthusiasm, and a willingness to learn, and a real passion for games. The other main trait we look for is communication: you’ve got to be able to talk to people. With a lot of applicants, their social skills are quite poor.”

Some employers are raising the bar, however, and looking for specific understanding of how games are made. “What makes CVs stand out is that they’ve either done games-related courses at university, or spent time working on mods or art at home,” says Steven Lycett, Sumo Digital’s development producer. “It is critical that testers describe issues as clearly as possible, which in turn entails a basic knowledge of how the games are pieced together,” agrees Arthur Parsons, an executive designer at Travellers’ Tales who started in QA himself. Anything else? “I suggest having problem solving skills, basic PC software skills, some schooling in an electronic field is a bonus too,” says Jai Kristjan, who worked in testing for four years before graduating to junior producer, and is now a designer at Vancouver start-up Slant Six Games. “And the ability to do a mind-numbing, spirit-crushing repetitive task for long periods of time.”

This is what surprisingly few prospective testers understand: that as well as playing the games within normal parameters, or indeed looking for bugs themselves, they’ll also be following test plans: performing mundane actions within the game incessantly, according to a timetable set by the designers or lead testers. “Very rarely is testing fun,” says ‘Paul’ (not his real name), a former tester now working in another area of games publishing. “You’ll tire of most games after a week of solid play. Walking into every single wall in a level to find map holes is just one example of a tester’s day in pixel hell.” Kristjan describes the day-to-day work: “Get a new build of the game, test an area of the game for hours, write up bugs, get a new build, regress your bugs, report failure/success… wash, rinse, repeat for months on end…” Some perspective is perhaps required, though. “Games testing, as a job, is quite possibly the best way to earn yourself £6 an hour,” offers up ‘David’, another anonymous retired tester, in perhaps rather faint praise. “It definitely beats stacking shelves, hands down.”

It’s worth bearing in mind that not every tester’s job is the same. Publisher QA departments tend to be large, brought in relatively late in the process and asked to do most of the grunt work. Developers’ test teams are typically smaller and handle a greater variety of work, though that will depend on how much the company chooses to rely on publisher QA. Testers at prestige studios like Bizarre Creations and Lionhead can expect to have some creative input: “[Bizarre’s testers] do far more than just test,” says Bizarre’s commercial director Sarah Chudley. “They’re involved in feedback on the design of the games and the balancing of the gameplay. They could be helping out with PR activities or working with the designers on new ideas.”
“People don’t think we’re creative in any way, but that’s not how Lionhead’s department is run,” concurs Robson. “We give a lot of feedback – we might have 100 suggestions about combat, they might use one, they might use all of them. But a lot of QA departments only see the game when it’s alpha, and it’s too late. They’re not really going to change anything.”

Like their development brethren, testers are no strangers to the nightmare of crunch. “The work load builds up towards the end, when the title goes to RC [release candidate] status,” warns David. “It’s at this stage that you may be expected to sign on to inhuman amounts of overtime. On some occasions, shifts are extended by a further eight hours, sometimes more. And then there is overtime to be carried out during weekends. It’s likely that you will not see a single day off in months. When you sign a contract, you may notice that you’re made to opt out of regulation four of the working time regulations, thus legally allowing you to do this.”

“I have one story to chill the bones of anyone interested in going into QA,” states Kristjan with, it turns out, justified confidence. “At one company, I stepped into a vital role as the only senior tester, and my managers did everything to keep me on site and organising the test team. My boss bought me a hammock, which he installed while I was at lunch, that I would sleep in almost every night. When I complained about needing clean clothing they had the secretary buy me a new wardrobe. A cell phone was bought for me, to get a hold of me whereever I was, even in the bathroom; when I complained about not seeing my girlfriend in two weeks – I lived with her – they paid for a hotel room for us twice a week to see each other. I did this almost straight seven days a week for seven months... when it was done I spent three months decompressing from the ordeal.”

It’s common for an aggressive, competitive atmosphere to develop in QA departments as testers look for any way they can to will themselves through these painful endurance tests. One programmer describes a test section he once worked in as a “testosterone pit” driven by intense competition on bug-finding leaderboards. Others resort to more extreme measures – there’s a story of the management at one developer/publisher requesting QA staff ‘bring their own mirrors’ after cleaners complained about the amount of cocaine they were having to wipe from the toilet seats. And yet it’s an odd fact that, at crunch time, testers can find themselves the object of developer jealousy. As measly as QA pay usually is, it’s usually hourly, and over 14-hour days it can easily outstrip that of salaried artists, designers and coders.

Creative staff are less likely to be jealous of the uncomfortable or downright unsafe working conditions testers can find themselves in. “The worst was being sent to help test for a development company which was not prepared for another person – they had my PC on the floor, in a closet without any light,” recalls Kristjan. But whereas that might be an uncomfortable one-off, David’s account of the test room at the publisher he worked at sounds like institutional carelessness on a surprising scale.

“Despite how big this company was, there were never enough games consoles for all the testers to work on. No effort was made to buy in more consoles to rectify the situation. The same went for memory cards; sometimes a whole team would be sharing one memory card for the whole of the day. Often you were provided with a broken controller that rattled, with buttons not functioning, exposed wires or a cracked, broken casing. We were generally working on 12-inch televisions with broken headphone sockets. These old televisions were also used for testing NTSC versions of games, but they didn’t support 60hz, so it meant playing games in black and white.

“Our health and safety was definitely not one of the company’s priorities. Wrist rests were not provided for those working on PC games day in and day out, and some people, including myself, began to suffer from the symptoms of RSI. Daisy-chaining multiple multi-plug adapters together was also common practice, which meant that when one fuse blew, the majority of the televisions and consoles would turn off, meaning people would lose their play-throughs. We were also made to use NTSC power supplies with the UK power outlets without using appropriate converters. Instead, we were shown how to take a ballpoint pen and jam it into the top hole of the plug socket to make the American two-pin power supplies fit into a live three-pin multi-plug adapter.”


You won’t find anyone who’ll deny that testing is an absolutely vital part of the process of game creation; but equally, and hardly surprisingly in the face of the above evidence, most admit that it’s seldom treated as such. “I don’t think there’s enough respect for test departments in development,” says Robson plainly. “Here at Lionhead we’ve got a lot of respect, they listen to what we say – we play the games every day, we know what we’re talking about. But on the whole we’re not listened to enough, without a doubt. I’ve worked with big publishers, and most of them hire just kids, that’s the trouble. Just for numbers. They’ve got 100, but we can do the same job with 20.” Chudley, however, has only praise for what she says is the professional and informed, and exclusively graduate, testing staff at Microsoft.

Parsons agrees that the problem is down to hiring practices, although he points the finger at pay: “I think that QA departments do get a bad rap at times, and testers themselves do not necessarily get the respect that they may well deserve. As a tester in this country the money is not great, which means that it can be tricky to attract the right people to it. It is usually these people that in turn make the development staff lose faith with QA departments. There was definitely a case of ‘them and us’ when I was in QA, and that probably hasn’t changed.” As a result, he has a strong personal preference for on-site testing: “It’s always a far more valid use of resources and is hugely beneficial to a title. Whereas sometimes, dealing with an external QA department, it can feel as though they are working against you, kind of the game police trying to catch you out.”

Testers talk about the same friction, though for their part it’s down to being scapegoats for buggy software, when in many instances the bug will have been found but simply not fixed for budget or scheduling reasons. “Only once did we let a major bug slip through in the course of many years,” says Paul of his former publisher QA unit. “A team that big – there were 50 – working that many hours will spot all the major stuff. Higher-ups then decide which bugs to waive and tend to waive more than we’d like, especially if they’re running late with the final builds. This attitude that buggy games are always QA’s fault pisses me off, to be honest. QA gets shit from every direction – the public, their wages, management, grumpy developers who don’t want to hear their game is broken, not to mention the work itself.”
It’s a bleak picture of overworked, undervalued staff and broken relations – though it must be stressed that the fault-lines exist mostly between developers and publisher QAs, and between publisher QAs and their employers, while in-house testers at developers fare much better. But ask whether these hurdles are worth overcoming for a shot at a job as a junior designer or producer – or even testing as a career in its own right – and even those who’ve suffered worst will defend it to the hilt.

“It aided me more than I can say,” says Kristjan. “It taught me every facet of the development process and allowed me the chance to hone my skills. I can’t think of anyone in development that couldn’t benefit from at least one project cycle testing a product to look at the project from a valuable point of view. In a funny way I went to school for four years and got my degree which allowed me to join the development team… only thing is, I got paid for that time, and have no scholastic debt.”

Not many developers claim to actively seek out QA experience when hiring creative staff, but most acknowledge that a significant proportion of their employees have such experience, almost by default: the combination of learning and networking opportunities makes the gravitational pull of the job almost irresistible. And of course, the chance to get noticed internally is priceless, especially within a company with a relatively enlightened attitude to testing like Lionhead. “It’s a good breeding ground for the company,” confirms Robson. “Quite a few go into production, junior design, scripting, art, animation. We’ll never hold anyone back in that way. If they can better themselves and the company’s confident, we let them go. Obviously it’s a pain for me, but it gives another opportunity for someone else. Senior guys like Peter [Molyneux] come over to us and say we’d like to hire this guy, and I’m like: ‘You bastards’. But I’m proud of it too.”

He is clear, however, that viewing QA as no more than a transitional purgatory on the way to greener pastures won’t help your employment prospects, and nor will it help testing escape its devalued status. “A lot of people do see it as a stepping stone. We try and drill that out of them, we want them to stay with us for two years. People that write that [they want to move up] on their CV, those are the people I won’t even look at, because they’re looking for a quick exit. I don’t think enough people stay in test.”

He may get his wish. For all that testing experience is still a common CV footnote, it’s increasingly just that: a footnote, a summer job done during one of the university courses that are squeezing out the opportunities for graduates of the games industry’s unofficial vocational school. “I don’t actually think that QA is such a good route into the industry anymore,” warns Parsons. “There are so many universities now that offer courses in game design, game production and development, alongside courses in specialist areas like coding, art and animation, that it is harder to get the break out of QA. Thatis not to say that it is not still possible, it is just harder now than it was seven or eight years ago.”

Perhaps that’s no bad thing. It’s beyond doubt – and to the industry’s immense benefit – that QA still provides an opening into videogames creation for determined, unrefined (some might say unspoilt) talent. But if the chances of success are a little slimmer, then there will be a greater proportion of games testers who aren’t necessarily looking for the exit as soon as they’ve entered. And when testers start to respect their jobs as potential careers, they may eventually begin to command that respect – currently so desperately lacking – from others.