Why is it that almost, if not all, of the recent large-scale missions (Vandal, Tech Smuggler and now Tactical Strike Teams) work fine initially and then become either bricked or bugged in such a way that they cannot be finished after a patch or two?asked by Janat928
TL;DRThey attribute recurring mission breakage to huge content volume, complex interactions, missing automated regression testing and a brittle Mission 1.0 toolset, and say they plan more automated testing and focused robustness work.
It's a good question. I had the same question when I joined the company a few months ago.
Well, why don't you take this one, Carl?
Yeah, I think there's two parts to this question, or two parts to the answer to this question. The first part, and I think the most pertinent part, is what Benoit, our CTO, mentioned, which is a huge scale of volume of content and a huge level of combinatorial complexity of that content with different gameplay systems and features, paired with a lack of validation testing. He talked a lot about our lack of a robust suite of being able to test, validate, pre-submit the work that somebody's doing, working fine for the content and the feature that they're working on, but is not going to regress something that was made seven years ago or what have you, right? Or even seven weeks ago. But that's the thing, right? So once something is released, it goes into the pot of this huge volume of content that we have. And just the scale and combinatorial complexity of all of that integrating together means that we run a pretty high risk of a change not being compatible with something that was released a while ago or fairly recently.
So what are we doing to stop it?
So essentially what we need to do is leverage more automated testing and more tooling around, whether it's gameplay feature development or especially our content development tools, to basically be able to sweep through a suite of all of our content and do some basic regression testing on all of those systems.
The next bit, I'm assuming you're going to talk about the declarative model.
Yeah. The other thing is obviously this person was directly referencing missions. Missions specifically have an additional challenge, which is the complexity of the tooling that we have around our current Mission 1.0, let's say, means that the resulting mission flows and the logic setups and all the things that we just talked about, like finding all the entities, collecting them all together, dealing with streaming race conditions, having to put things into the mission logic to deal with things like server crash recovery and things like that, it results in a very brittle system.
I mean, a good example of this going wrong in some respects is Race for Stanton. What was interesting about that was, for some reason, the way that the mission system interacted with the back-end systems, it funneled everybody to this prologue mission or setup mission to one location. And nobody wanted that. It didn't happen in our own testing because we didn't have enough concurrency. And then when it goes live and the live environment is slightly different, it then funneled everyone, and it's like day one, oh. What? And it makes it poor for the player. It makes it a poor experience for us, because people have spent a long time trying to make the content and get it out live. So that's something that, none of these issues do we want. We want to be adding new content to the game, and every patch should make the game better than previously. And that's something that we're very well aware of and we want to improve on. And we have taken steps, like you said, in terms of refactoring some of these large systems, but we've also taken steps into highlighting these individual areas within the game or development in terms of buckets. So we've talked about the first hour for a new player. We've talked about quantum. We've talked about freight elevators, hangars, and taking those and really doubling down on making sure that they are robust within a short time frame. So yes, I think that is a great question. We're very well aware of the status of the game. Our community is very vocal and we are thankful for that. We want to say that we are taking that on board, and absolutely we will be improving on that going forward.