George H

I've landed on this one. So like Sam said, I work on the core team, which is kind of like we check the core mechanics of the game to make sure that when all the new and exciting stuff comes in, we haven't accidentally introduced a new variable that completely ruins the game somehow. So I'm kind of there on the front lines almost of just making sure that the game in its base form is still playable. So in core team, I'll have a series of checks and we'll cycle through them. So we'll spend a couple of weeks making sure the player is working. Then we'll spend a couple of weeks making sure that all the different landing zones like Orison, New Babbage, Area18, they're all still working properly. We might check missions and we'll go through all of those. And once we've finished, we just loop back around to the start and start again. So a usual day for QA is I'll come in, the rest of my team have usually already started because I enjoy a lie-in, and we'll spin up our own sort of microcosm, a little tiny server that we just test with these six or seven people we've got, and we have a big long list of hundreds and hundreds of checks that are all individually very basic. So it might be like you're going to Port Olisar and it's like, are all the textures correct? Is the atmosphere correct? And we'll just systematically go through all those checks for whatever section of the game we're looking at for that week. And anything that's wrong, we'll sit down, write it out, and then pass the information up to the developers. And then it's kind of their job from there to fix it.

You said with the six or seven people, that's not the sum total of everybody in the QA team?

No. That's very specifically seven, eight people on my team just looking at the core features. Again, there's lots of different teams of people looking at other areas of the game.

I'll add a little bit from the network side as well. So kind of similar to Jack, but obviously because I'm on the network team, I'm just testing network-related things and persistent tech and stuff. And one of the caveats for me doing this was I have to big up the network team and how great the network team are, which you guys are fantastic. So hello, if you're watching. Thank you, network team. Hello to Clive if you're watching. So my typical day, I'm usually an early bird, so I'm in at like seven o'clock. If we're continuing with a task that we've been set beforehand, we'll just crack on with that, check things like emails and messages like normal. And then usually around about 11:30, we'll have a stand-up where the team will get together and have a video meeting and say, this is what we're doing today or what we've done for this morning, and this is what we're now doing for the rest of the day. If anything major comes up, we can point that out in a meeting, say, we're testing this build, but we're having this blocker issue which is preventing us from doing it. Because we talk to the devs directly, it gives us more of a frontline contact to say, this is the issue that's blocking us, and then we can go away, get any evidence that we need, any videos, logs, all that sort of stuff, pass it on, and then we can basically just continue with our day. And then usually about four o'clock, I'm heading off for the day.

Not at all. Definitely not. When it comes to devs and QA, obviously, we are there to somewhat ruin the day. And we do come in, and sometimes there'll just be something that they have not expected that crops up. But the vast majority of the time, I think we have a pretty good connection with the devs. None of them have called me names to my face. But yeah, we work very closely pretty much from the beginning of when a feature is being designed. So for example, at the moment, in the run-up to 3.15, we've got the healing and inventory changes that are coming in. We've been working really closely with the Actor Feature team on that. Guys like Chris Parry doing absolutely wonderful work, and they guide us and we help guide them. So we're all striving towards that same goal of having a good quality product. Developers also test their own work. So it's not just us that are coming around and saying, "Hey, it's broken." If they go through and launch it up and something's broken, they also don't want that. So yes, as much as it is kind of like a Tom and Jerry cat-and-mouse comedic relationship in theory, in reality we do have quite a good bond with those guys, which is really fun. And honestly, QA gets to touch on so many different areas of gameplay that we do have a close connection with all of the devs. Say, for example, artists may never get the chance to work with a particular area just because of the nature of their job. So it's really great to have our tendrils in there. And George being directly in the network team, obviously, if you didn't have a good relationship with those guys, that wouldn't work so well. I'm hoping stuff's good.

Yeah, stuff's going very well.

Also, because we touch on everything, we potentially know a bit more about other features than the devs, because they'll be very focused on what they're doing, their one little thing. For example, back when I was doing UI stuff, when I was here before, there was an occasion where literally they did something and it impacted on another system they hadn't even considered. The second we saw it, we were aware and we obviously fed that back. So having that relationship with the teams, you've got the ability to give them knowledge they sometimes don't necessarily have access to, because they're so focused on their one little area.

So like you say, we do have those main QA teams, like Jack's team, like George's team, that are the main gameplay-focused QA guys, and they run through together as a team of QA. Granted, they still do contact developers and work with them that way, but they're more of a QA team. So we've got those main QA teams in the UK and also in Austin, Texas. We have a few testers who work as part of main teams like that, as well as the core and the gameplay teams. We also have a vehicles and FPS team, which I led for a short amount of time. It was really fun to delve into the ships and the shooters kind of stuff, basically. And then we also have the tech team who deal with everything editor-based. So all of the tools that our developers use that also need to work. So not just the game-playing aspect of things, also the game-making aspect of testing too. But yes, like George here, we have multiple embedded testers who basically are part of their teams directly. So George is part of the network team. We have people in the UI team, audio team. And basically they get to be the person who is the point of contact who knows everything about that particular area. When it comes to QA, we are generalists. We try and do our best to provide info across the entire game. But obviously we can't remember everything. There's a lot of stuff in terms of logging, debug. I can't even get started on crashes. There's so much with crashes. We have a crash guide that is literally an hour read on its own, just all different sections. So yeah, it can get quite complex. And having those embedded guys who are literally there to know the exact ins and outs of everything to do with that particular team is so, so useful, for general QA and for the developers themselves. It is a really, really good kind of symbiotic relationship, I guess, that we have going on. George, do you want to talk a little bit about your transition over to embedded?

Yeah, so it was kind of like a weird culture shock in a way. So before joining games, I worked in retail for almost 10 years, did my degree and then started in QA. Moved through a few different departments: before, I was on live release, so what gameplay is now, so we'd test the features before they go live. Then I moved on to the core team, which Jack is on now. Then I moved over to tech, then I moved over to the resource team with Sam for a good couple of months, and then I ended up moving on to the embedded team. So it was quite a weird culture shock, because the first thing that I thought of going into an embedded team was, I need to make sure everything's good. So if I see a texture bug, I need to bug it up, which, that is the premise, but usually when you're on an embedded team, you just focus on the stuff that your team does, because that's why you're there. You're running around trying to bug all of these things up, and it's like, no, just take a minute, calm down, bug your network stuff up, and then relay that to the devs. And that's kind of the way the team works. I think the biggest thing is that when you're a generalist QA, you're not really talking to devs on a day-to-day basis, really, because you're more relating to your seniors or your leads. Whereas going on to an embedded team, you're talking to your lead, who's obviously an engineer or developer, plus your seniors and the junior programmers and all of that on a day-to-day basis. And when Sam was saying before, it's like a catch-22. So when people say, oh, devs must hate QA because they're always finding these bugs and can't they just stop so we can do other things. But at the end of the day, it goes hand in hand. So I'll go to my lead or my senior and be like, oh, I'm not too sure how to use this specific tool or command or something. And then they'll come back and be like, oh, it does this, it means X, Y, Z. And kind of the same vice versa as well. So if devs are in-game and they're not too sure of a specific thing, because a lot of the things in-game do change quite rapidly as well, because the game is huge, then it comes to us: I'm not sure how I do X, Y, Z. So then we'd go and help them do what they need to do in order to check their fixes have taken, basically.

I think we've somewhat touched on the aspects of it from the day-to-day stuff. But yeah, like George says, it's the different dynamics of we have QA teams and then we have our embedded guys. And then we have the Squadron team, which is kind of a hybrid between embedded and also being its own small team of QA, and those guys did great work as well.

That's totally fine. I understand. Yeah, so actually, automating testing for a project like this is quite challenging, to say the least. As it stands, we are moving slowly towards being able to automate certain test cases. We do have a guy who is embedded automation, actually, and he is working slowly towards making it more available to all of us. But we do have a couple of things that we use internally already. George, on your time on the tech team, you might have touched on some of this stuff with TestRunner.

Yeah, that's correct. So TestRunner would grab a build and run through basic sanity checks. It doesn't go too in-depth, because obviously when you're making an internal development build and then running it through TestRunner, it can take a few hours to do that. So it goes through the basic checks. And when we try and grab a build, it'll give us all the information: this was fine, this was fine, this was fine, but this was broken. Sometimes we can get around it because it's not a core thing that would stop us from testing. But there have been cases where we tried to grab builds and it would just break continuously. That's where TestRunner picks things out, so we can identify that before we go into game, to say, we need to do this task, but this build is broken, what do we do? And then usually the devs will go and fix it ASAP and kick a new build, or we just revert back and then carry on the testing.

Yeah, exactly. It's mostly launch sanity checks and that kind of thing. Very basic pass/fails at the moment. But we are working our way towards it, and it is something that we are very interested in. So if you are an authority on automated testing, please do come in. We would love to hear your opinions.

I think Jack can answer this the best, to be honest. You deal with the most crashes on your team, I think.

The stream I work on is notorious for being potentially the least stable one in the game. Patience is obviously a big part of it, but I think the thing I've used the most since I moved on to the core team is my memory, remembering things. A lot of issues tend to maybe disappear and then come back. And after a while, you just remember the really bad ones. So something might appear that's been gone for three months, and you can just look at it and go, oh, I know exactly what that is, I've seen that before. And you'll go and find it, and you'll link it back up, and you'll escalate with the devs and go, this has started happening again. What did you do, please? Please help.

The other one I'd say is curiosity, quite a bit, because you find something which isn't quite right, and you pull the string, and then you find something else, and then you find something else, and then you find something else.

I think that is a lot of what a lot of people don't really understand about QA sometimes. There's one thing, and you think, oh, that's fine, it's an easy fix. But then that will branch off into 100,000 different things that need to be fixed: so does that, so does that, so does that, before this main thing can be fixed. It branches off so much. Game dev is so complex, it's crazy.

Oh, I would have to say the amount of things that we learn. We learn so much in QA, whether it's parts of dev and just the game, and just generally how games are put together. I love learning things. So that's what I love about QA. It's just being a sponge of knowledge.

For me, there's a lot of different things. The thing I think I enjoy the most is the people. It's such a good, vibrant group of people who, even when you're having the horrible crashes, you'll be able to have a joke and have a laugh. And if it wasn't for a good group of people, I don't think I could do it. But it's such a good group of people.

I should have gone before George, because I was going to say the same sort of thing. I think, especially on the game dev stream, which does tend to have some minor issues with stability, you do form a real sense of camaraderie with the people around you, because it's not just you that's having a horrible time, and you've been trying to get into game for four hours and you've not made it past the main menu yet. It's you and seven other people who are also having a horrible time and haven't made it past the main menu yet. We're kind of friends for life, forged in that fire together.

I guess as well as being able to do the same things as those guys, I love learning stuff. I love picking the brains of all the different developers, especially the big-brain people like the directors who just know so much about how to make everything cutting edge. I guess just generally being able to work to support games is something that I really enjoy. When I came out of university, I did an animation degree and I hated animation. I came out of it and I was like, never again in my life. But I knew I still wanted to support developers, and I knew I wanted to support games development and do that kind of stuff, because that is something I'm so passionate about. I didn't have a clue what I wanted to do, and I just fell into QA by pure chance, because it was something that was entry level and everybody could have a go at. And I ended up staying here because I love it so much. So yeah, I think it is something that really is an interesting career, and really is a way to get your fingerprints on a game and put your influence on a game's development without having to necessarily be a creative or something along those lines.

Oh, crashes. Crashes are the main bugbear, especially if you're working on a test, you get right to the end and the server's gone and you're like, no, why?

Absolute pain in my life. You're just trying to get something, you've been all set up, you've been with somebody else, multiple people, you're just set up to get something tested, and nope.

Yeah, you're managing to hook up your thing, you're about to catch the bug that you've been hunting all day, and then the pop-up.

A completely unrelated crash.

Yeah, completely something else, and you're like, no, I don't have time for this. It's never the one you wanted. Yeah, I guess that would probably be my least favorite aspect: when bugs pop up in the search for other bugs, and you just have to go, not now.

Shh. Go to sleep. Add it to my Post-it note of things I'll have to look into later.

The good old sticky note of unloved bugs.