Back to search

Server meshing: past, present, future, static, dynamic, 3D. Talk to me about server meshing.

TL;DRStatic server meshing has run for months, with the configuration changed every patch, hybrid performance and binding improvements helping desync, and lessons about load distribution informing dynamic server meshing.

Benoit BeausejourChief Technology Officer

Well, so obviously we launched server meshing at the end of last year and we've been running a server mesh environment for now seven months, eight months, nine months. We're nine months already. And so that's the static server mesh. And so that means we have territories. Just a quick recap for everybody. That means that we have a configuration for these territories. And we can pick different top-level object containers from the tree of the universe. And then we basically distribute them on different resource servers. So at the moment, we run a 10 game server per shard setup. And so that means that we basically distribute the key areas on these 10 game servers. And that's how, when you transition from one zone to another, it's seamless. But you're moving from one server to the other. So we've been kind of taking the year to get used to being a server mesh environment. And so that means we've discovered a bunch of gameplay bugs related to this. I think we talked about some of those in January that have been solved. For example, the quantum drive systems didn't properly handle the transition. Sometimes we'd lose power in transition, and that would cause the thing to reset, pull you out of quantum, things like that. That's one example. But there's a series of other issues that were related to that. But more importantly, we've been also learning about how to properly distribute the load of the simulation. So now we have this tool, we have this map. It's not the end of it right now. Every patch this year, we've changed the configuration. I don't think we've had a single static configuration since the beginning of the year. Every patch has been different. And so obviously, we've tried to make sure that we dedicate resources to where there is an event. So if it's ASD locations or Hathor locations or Stormageddon locations, we've kind of reconfigured the mesh at each release to basically guarantee some server power at each place, because we know that people congregate in these areas. We've also learned a couple of things, because the teams that make content don't necessarily always talk to the team that makes the tech. Where you put those locations or how many there are is important. We learned that through the Hathor moments, where they were a bit too close and we couldn't distribute them properly because of other limitations, which I'll get into in a bit. And so we've changed that config every time. And so it's been really a good learning experience for the team. We've also solved a bunch of issues in the static mesh related to performance. We talked about stability of game servers earlier, but the hybrid is at the center of this. This is the networking component that you're connected to when you're playing the game. It's doing the replication. It's doing the binding and all that stuff. For this guy, the stability score is zero. Meaning if there's one crash, we don't ship the patch. That's it. That thing needs to run perfectly at all times. And the kind of quality metrics that we expect from the hybrid are very, very intense. The speed at which they transfer packets or accumulate packets and transfer them between areas and elements has to be very, very fast at all times. Or else you get rubber banding, you get desync, things like that. And so there's been a lot of performance improvements behind the scenes. Especially, I think, just before Stormageddon, or is it after? At this point, it's a bit of a blur. But I know that we found a couple of key optimizations that have really helped the hybrid maintain stability across some of the big binds that we were seeing. Again, binding is this process that when your client needs to bind to an area, it means it's receiving updates for an area. This whole binding process is a bit costly on the hybrid, especially when there's big areas being bound. And so optimizations have gone into that. And that has helped desync quite a bit. There's always room there for improvement. And the team are still actively working on this. That's never going to be the end of it. But it's much improved since the release of 4.0. I hope the rubber banding is getting a lot better every patch. We're hoping to always cut some of those big bottlenecks as we go. So that's static server mesh. We also learned a couple of things on how to distribute those locations. So the cost of a location and the amount of players is not necessarily the same weight. Funny enough, a lot of players in the same place, let's say you have an empty room with nothing in it. And I can pack 300 players there. No problem. Server's good. No problem. If I add AI in there, we start getting into issues. If I add ships in there, bigger issues. So the cost of the location, not necessarily multiplied, but factored in by the number of players there, is kind of what the cost of an area is, and how we should distribute the computing power that we pay for to run the game. And so we've been learning a lot about this in all these experiments we've been doing, and that's informing dynamic server meshing, right? Meaning what the algorithm needs to be in order to properly distribute the load of the simulation to the different components in the game that can do the replication.

Star Citizen Live›Tech Talk w/ Benoit Beausejour·
Published on this date.11 Sep 2025
EpisodeServer meshing