Star Citizen: Calling All Devs - Of Golf Balls and Tessa

Well, it's an entertaining bug that's happening. The root of the issue is that the persistent data, the persistent loadout for players, is getting wiped out, and effectively the body is getting detached. And when that happens, you end up spawning as a little white golf ball and bounce around. This is not the first time it's happened, but it does surface from time to time from similar root issues. And in this particular case, it got out in 3.0 and started happening to a lot of people. And we weren't entirely sure what was causing it this time. After some investigation from QA and from our LiveOps guys, it looks like they got a solid repro for it, which is: in a heavy server, making changes in the Personal Manager app was ending up sending incomplete or missing data to the persistence database, which was causing this kind of corruption. This is one of those situations where obviously we did a ton of QA and bug fixing in persistence before we went out and we just didn't see this at all. I fixed a lot of persistence bugs and this was not one of them that even came in. And the reason why it was showing up in live and not for us is because of the much heavier traffic and performance issues at scale. And so we were able to find, thankfully, again with some hard work from QA, some reliable repro steps. And they were able to reproduce it by having people on the server and then also doing some fast swapping in and out of mobiGlas, applying changes in PMA, and doing this kind of whirling dervish act, you could eventually arrive at this white ball.

Now, you said it wasn't seen in Evocati.

It was seen in Evocati. Too infrequently for us to find the source for it.

This was one of those great moments where it's always important to remember that every Star Citizen environment, whether it's the Evocati or the PTU or even our live alpha environment, is still a testing environment. They're all testing. And people are like, why did you push 3.0? Because some things we knew we wouldn't be able to track down until we got this out to as many people as humanly possible. And it wasn't until we got it out to everybody, and started seeing the hammering that can only occur when everybody is in there, that we were able to actually track this down.

It's a little bit up in the air, actually. We put in an initial fix for it that tried to prevent the PMA from making alterations when it's in this uncertain state. Effectively, the PMA operates using a lot of asynchronous operations. We need to make a request to the server to get the current persistent loadout of the player and all of the items that could potentially go on any given item port. Additionally, there's many chains of asynchronicity in the code itself. So you have the Flash on the front end, but then that needs to talk to something in code. And so we have the Flash, which talks to something called the context component, and then it itself fires events to something called the provider component, which then coordinates using a bunch of remote methods to send requests to the server. And then we get responses and we change back and forth. So there's lots of opportunities, when you have these kinds of timing operations, for things to go awry. And so clearly we were playing a little bit fast and loose in this particular scenario. And we tried a couple of solutions to try to make a small change to fix the problem. You know, 3.0 is out. We don't want to make some large change to the code to try to fix this bug and then introduce four or five more. So we tried small changes to fix it. They started to fix it, but you could still get it to happen with a little bit more dedication. And then QA was yet again able to find a more reliable set of repros because they were looking at it longer. And this even led to finding some other bugs that we didn't even know were there, like spawning small ships in the actual game. We won't get into that. So anyways, long story short, long story long, we ended up doing a more comprehensive set of fixes. I helped a bit, but a lot of it was spearheaded by the UI guys in the UK. And effectively, there's a set of changes that probably fix it, which is: don't make changes to persistent data when you know you shouldn't be. So if you haven't actually changed anything, don't allow a save. Don't allow saving data when you're still waiting on something to finish. So if you've made a request for some data and you haven't gotten a response, don't save the data. And also prevent immediate transformation of the data. So whenever you open the screen, or after you've made a change, you shouldn't be able to, the next frame, again apply a change to that data. So basically it's reducing all these opportunities for incoherent data to make its way in when things are still in motion. And it looks like we've got mostly a fix, and I need to look at my email and see what QA is saying right now.

Yeah, this is a fix that's currently in testing in a 3.0.1 patch that's with Evocati right now, right? So we are currently testing potential fixes. I know we rolled out one version of 3.0.1. We're getting ready to roll out another version of 3.0.1 soon. By the time this airs, we'll probably have done one, maybe even two more.

I mean, I've helped with a lot of other 3.0.1 patches as well, and this is kind of the last one that won't [die]. But I think maybe we've got the bullet for it now.

Not a lot of active progress. It's definitely something that we're keeping in mind. It's something that we definitely want to deliver some fun options and choices for people to make. But in the reality of things, getting more ships out is the bigger priority versus getting these little tchotchkes in. But we've definitely got a few lists of different dashboard decorations, almost, where it's like, yeah, I know, because one of the old stretch goal ones was a bobblehead of some sort that everyone wants to see. So it's like, okay, not just a bobblehead, but what if it just takes general physics? Can it shake with your ship? What are the kind of weird little things, the pendulum swings, whatnot. So there's definitely a lot of stuff that we want to explore, and it's just in its holding pattern right now, waiting for that right time to really build everything out and then start getting all of that integrated into the game.

I remember, gosh, it was a couple months ago, I guess, you were going around the whole office and asking people, what would you like to see in your spaceship? Just taking ideas. This has come up a couple of times. I know some of the guys on the Flair team in Austin have asked questions about internal ship decorations and stuff like that. So it's definitely something that's on our minds. It's just, obviously, one of the biggest aspects of game development is prioritization. So it's something we definitely want to get to. We don't know if its time is now just yet.

Yeah, because also once we frame some of that time and we really start diving into this, getting a lot of that other work out of the way first is going to let us really dive into it. So instead of just being like, oh, we've made a bobblehead or something, it's like, oh no, we've got a whole run of bobbleheads. We've got all these other little tchotchkes, you know, bring your fan back and attach it to the sidewall of your ship or whatever.

Just an idea, Cutlass people. He's not committing to it.

Hey, no, we did bring the fan back. The fan is on the pilot seat of the Cutlass. It's just an integrated little box fan now and not a have-to-punch-the-window-to-make-it-work fan.

To be honest, it's not really a priority for us this year. We know it's a super popular ship. Obviously, it came out of The Next Great Starship. Big fan favorite, but it's not where we need it to be. It has a lot of design problems, and it's not needed in the PU this year. It's not needed in Squadron this year. We have ships that we need more coming, and it's prioritized appropriately. We know what we want to do with it. Just to quickly go over some of the design issues with it: there's the mystery ghost train turret, where you sit in a seat and you go outside into space to get into the turret. It's a dropship that holds six people, which is not particularly fantastic. It's got lots of tight, twisty spaces that just aren't in metric. So bringing all that together is a fairly hefty rework. And we've got two ideas internally where we want to go with it. It is half a gunship, half a dropship as it is. And basically we're looking at those two avenues of just going full gunship or full dropship. At the moment, we're leaning more towards the gunship side. We've got other dropships that are better dropships than it, but we don't have that AC-130 style gunship that I think a lot of people would like.

Now, let's clarify that for a second. We explore lots of ideas. We have lots of conversations. We consider doing this. It doesn't mean that's necessarily what's going to happen. The decision has not been made to make it a gunship. The decision has not been made to make it solely a dropship. It's a conversation that has to happen. This was a ship that was made by backers, and like I said, outside of the metrics we had, and then we've changed the metrics since then. So it does need a lot of work. And there will be some changes to it when its time does come, but it's not its time yet.

It's bubbling away in the background.

Yeah, it was actually pretty hard ripping her out, actually. She's our first mission giver. But yeah, she's coming back. We've actually recorded her voice for 3.0, because we had all the actors in again. It would have been great to get her in for 3.0, but we had so many other things. We had Miles Eckhart, and Ruto was even a stretch for 3.0. We've got her in the back pocket, as it were. The time I want to bring her back is when we have the missions that will do her justice, which is when we get scanning in, because she's the perfect mission giver for scanning-type missions. And I do know that the universe is a little more hollow without her. Honestly, there's something about her actor and the way that she was written that really just makes her sparkle. So yeah, don't worry. She should be coming back. I do just want to do her justice, though. I don't want to bring her back to just have the same fetch missions that you've seen already.