Back to search

What are the short-term plans for dealing with server desync in the PU and Arena Commander?asked by Breezy

TL;DRShort-term, they are optimizing server performance and thread contention, but the real fix for desync is a new network serialization system, with spawn bundles and sending only changed data, arriving with Item 2.0.

Chris RobertsFounder, CEO and Chairman

Well, short-term plans, we're trying to address them. So part of the issue is... I mentioned this in the 10 for the Chairman that I recorded yesterday, but no one's going to see it until next Monday. But one of the reasons why there is some desync is because the server and the clients don't run at the same frame rate. So we can quite easily have a server come under significant load and drop all the way down to five frames a second, whereas someone could be running on their machine at 30 frames a second or 40 frames a second. And so one of the big things that we're trying to do is fix some of the server behavior. In the last few patches, we've added some more functionality in places. The more AI you fight... the AI is all simulated on the server. That actually stresses the server more than it would stress the client, because on the client side, it simulates your local [ship]. The remote clients all come from other clients that pass up their information to the server, and then the server simulates all the AI. So every client normally does the IFCS simulation for one ship, but the server will do the IFCS simulation for all the AI ships. So if you suddenly have 50 or 60 or 70 AI flying around, that's 50 or 60 or 70 IFCS simulations, and the IFCS flight model simulations are pretty heavy mathematically. So we're looking at some stuff to optimize how some of the stuff is done, because more and more functionality has been added to it, but we haven't really spent time optimizing it, and that's one of the reasons why the frame rate goes down. Then we have some thread contention issues between network and the physics and the main thread on the server. And so we're just trying to fix and address those to get better server performance, which will help with the desync you see. But the real answer is the full-on back end, the full-on new serialization method that we're going to use, which completely junks the old CryNetwork setup, which we're fairly far along with, and that will be switching out. And that hopefully will just make things much... there'll be less bandwidth going over the network. Because right now there's a lot of bandwidth. When you spawn a ship, there's a massive amount that goes. And also it just ends up sending more data than it needs to. Like if you have a position change, well, the way it's currently set up is these things called aspects. And there's only a limited amount of aspects that can be sent for any ship. One aspect could be your movement, which would include your position, your velocity, your orientation, a bunch of other data. And so that could be, I don't know, a 60-byte or 100-byte chunk. But if you just change the position, it has to send that whole blob of data. But really, you shouldn't have to send that much data. So our new system isn't limited that way. It basically only sends the bits that change. And it's not limited to 32 aspects or anything like that. It's much more flexible. And so that will reduce the data. The other big thing that would reduce the network data is when you spawn a ship. We have very complicated ships. So you have even a Hornet, which isn't a big ship. So imagine a Constellation or something bigger like a Starfarer or Idris. But say a Hornet is a vehicle and then it's got about 60 items attached to it. So what happens right now is the server will spawn the vehicle, the Hornet hull basically, and say, spawn this to all the clients. So that message goes out to all the clients. And then one by one, it starts adding all the items on and sending a message saying, spawn this, spawn this, spawn this. So you're basically sending all this traffic about the Hornet to spawn, and it's going back and forth, there's a lot. And the simple thing would be just to be able to send a message to say, I'm a Hornet, here's the 60 things attached to me, you go spawn it. Rather than one by one, spawning, doing a handshake between servers and clients. So that's what we call a spawn bundle, where you basically just deterministically say, here is the ship and here's all the entities attached to it, you go build it. And then the remote clients are all just doing that and it all happens. And that would greatly reduce the data, because something like a Retaliator can end up being about five megabytes of network traffic just to spawn a Retaliator. Whereas in the method that I'm talking about, you're only going to be passing your entity ID. So in the case of the Retaliator, I'm not quite sure how many things are attached, but that would be the equivalent of, say, no more than 150 times 64 bits. So basically, a lot less. Don't ask me to do math. So those things will [address] the desync problems. And we're aware of them. So what we're trying to do is fix what we can on the current system, because our new system's not that far off from us starting to deploy it. So we don't want to do too much work fixing the old system, because the new system is going to be much more flexible. But the new system comes in with the Item 2.0 stuff. But yes, we definitely want to address and fix those issues. So that's what it is.