After 3.23 goes live, what metrics will you be watching to determine which portions of master modes are successful and which may still need additional tuning?
TL;DRAfter release they will look for player feedback on how specific ships fit their roles, to deepen the master modes framework, and compare it against recordings, kill-death ratios and accuracy data rather than any single source.
Usually what I'm looking for to pass on to Yogi and Rich is ways to deepen what we have. So what I see of what we created is a framework. And so now once we have a framework, we need to find ways to add depth to it later, to add individuality to these individual archetypes and these individual ships. And so that's the kind of feedback I'm looking for: somebody specifically talking about, hey, I'm trying to use my Khartu-al, my San'tok.yāi, and I'm finding it not very effective in these specific ways, and it doesn't match the role as described. I'm looking to pass that kind of information off. Or I'm using my Vulture and it's basically impossible for me to get out of a combat scenario. I never do. I'm always trapped into it. Those kinds of things where players are presenting problems that I can then pass off to the team with context and go, okay, this is what we're seeing players observing. Does this match the design we're going for? Maybe we need to make some adjustments there. And it's that deepening. How can we add further depth? How can we add further skill checks? How can we actually make this a little bit better of a skill curve, a little deeper, all that kind of stuff. That's usually what I'm looking for.
But just to add on that, there was some more to it, right? Because what we're also getting from Michael's team are sometimes recordings of actual flight combat that happens, which we can then analyze and go back and forth. We're getting KDR graphs and so on. So sometimes we see that something is wrong in the data. There's a lot of stuff that is going on. It's not just the qualitative stuff that Michael just described. It's also quantitative data.
Yeah, that's an important part. I do want to mention that we don't just look at Spectrum and Reddit or YouTube videos and go, this is how it is. We compare what people are saying with what we observe in the data from our behavior analytics, too. We look at kill-to-death ratios. We look at accuracy values. And we combine that with our own observations in the game. So sometimes people might be like, multi-crew ships are terrible. Then we'll look at the kill-death ratios and we'll go, well, looks like they're doing really well. And then we'll go into the game and look and go, wow, everyone just avoids them. And that provides that kind of context. We don't ever look at a singular piece of information and go, this is the truth of the experience. We look at everything we possibly can to provide all of that context.