Ahmed Shaker · Senior Director of Publishing Technology

So talking about what John was talking about, about how games push boundaries, that is so true. And it's one of the areas that we're always chatting about; it always comes up. Like right now, everybody's obsessed with AI. When you think about it, AI is powered by GPUs, and GPUs came to exist because of video games. So kind of a similar analogy: when 3.18 got published from PTU to Live, a lot of players did not know that the typical ratio between PTU and Live is usually 25x. The problem with 3.18, though, is we decided...

Jared Huckaby · Host, Community

One of the problems.

Ahmed Shaker · Senior Director of Publishing Technology

One of the problems, that is true. But it starts from the fact that our universe, with all of our moons and planets and players and ships, we decided to persist everything we can as often as we can. And that is a recurring theme in Star Citizen. A lot of players might not notice that we challenge ourselves as early as possible, instead of waiting till we reach the million or the two million or the three million concurrent players. So we try to amplify the challenge all the time by avoiding any kind of hacks or shortcuts. And you see it in every single aspect, but I get to see it myself as well in my field. So I still remember when we first published, our public cloud providers reached out to us and said, what is wrong with you guys? Like, what is wrong with us? And they said that we're moving an insane amount of traffic between zones. Some of our backers are technologists, and they know when you deploy resources in the cloud, you have to split your resources across zones. You cannot sit in a single zone because zones can go down, and they do go down quite often. So the first few days, and the first few days, by the way, we were not peaking concurrency at all. A lot of backers chose not to play the first few days. They like to wait, because a lot of them learned that we hotfix a lot of issues behind the scenes. We were pushing 1.2 petabytes of hot traffic every single day, and that's between zones. That's not between every single server. We still have traffic inside the zone.

Jared Huckaby · Host, Community

You said 1.2 petabytes?

Ahmed Shaker · Senior Director of Publishing Technology

Petabytes every day, between zones, in one region, not in all of our regions. We exist in four regions.

Mike Jones · Vice President of Corporate Technology

And that's just unheard of.

John Erskine · Head of Publishing

Yeah, the Amazon guys were like, we've never seen this before. Not like you've gone up from what you're used to. They were like, we don't have any other customers that are pushing this kind of traffic.

Ahmed Shaker · Senior Director of Publishing Technology

A day. And when you think about it, like social media: a human being has to create content, the content has to be uploaded, and it has to be interacted with. And most of that is happening in eventual consistency, running through compression. The fact is that every single simulator that we have... and right now, while we have instances with a reasonable amount of players, we have copies of our universe. All these copies are producing data as fast as the system can handle. And we are a competitive group of people, so none of us is telling everybody else, hey, please slow down a little bit. No, everybody's saying, give me more. And that goes all the way down to the wire, to the database. And I know a lot of players, whenever they hear the word database, they get really upset. The bit that I would like for some of them to understand is we really don't want to hold our innovation behind the scenes for too long. They get to see right now we're testing Server Meshing and server crash recovery in Tech Preview. Each and every one of us cannot wait for that to go to Live. So 3.18.0 is one of these stories where we could have stayed longer till we figured out every single little problem that could happen. But they were hungry and thirsty and really very eager to play that version of the game. So thanks to all of them for staying with us till we figured some of the kinks out. We still have a lot more to go, but we will keep pushing, because that's what we do. That's why we're all here.