TL;DRMap streaming is more common in single-player games because most multiplayer games keep maps small enough to hold in memory, whereas Star Citizen's maps are vastly bigger so streaming must work online without stalls.
Well, map streaming is generally more common in single player games. A lot, not all, but a lot of multiplayer games will confine the maps to smaller sizes so that it can all fit in memory all at once. So once you've loaded in, all the data is resident on your client, no streaming's required, and that cuts down the chance of any stalls or any frame rate hits and makes for a smoother online experience. So the challenge is to do streaming during a multiplayer game and not have any stalls or performance hits. And it's also, as I say, about the scale that we're doing this on. Obviously, our maps are millions of times bigger than the maps of virtually any other game. So it's a question of scale and trying to do it while we're online as well.
Yeah, we're probably one of the bigger games ever envisioned at this point. We've done quite a lot of stuff over the last couple of years where we've had snippets of gameplay, Star Marine, where it's been a small contained bubble of the game, and we're now going, right, it's now that, but it's in this big massive universe that's making everything bigger. And by going, well, if we only need to stream in this one bit where the FPS is happening, the performance should be the same as Star Marine, probably, on that point.
TL;DRBind culling unbinds far-away entities from the client, which breaks things like markers, entity pointers and scanning, so lightweight streamable markers were added and much gameplay code now asks the server for results.
So this is the act of unbinding the entities on the client when they go out of range. So if you're too far away from stuff, it basically streams it out, is the idea. There's quite a lot of implications of that for gameplay code, which I'm obviously slightly concerned with, being a gameplay programmer. So it's the type of thing of, if I'm on one side of the universe and I've got to go and kill someone on the other side of the universe, in the game just now you'd normally see a little objective marker, but that's not there because he doesn't exist on my client, because he has been unbound. Or simply if you have a comms call with someone who's too far away, he'll be unbound. Or similar things. And from a more technical standpoint, all our entity pointers themselves will, as soon as something goes out of range, become invalid. You can't talk to that entity anymore because it doesn't exist for you. You can't get his status, you can't get the amount of health he's got, because it no longer exists on your client. And you've got to be able to handle him leaving and then coming back again later on, so if you've got to fix stuff up, and if you've then got to rework your gameplay system to make it work with someone that's just come back. So what are the answers to those things? So for the likes of the markers, they were solved by having an independently streamable object. That's basically saying we've got this little marker, which is a really small lightweight thing. It's really far away from you, but the server's going to tell you that it has to exist on your client because you need to know some stuff about it. It's removed from the normal hierarchy. If you've got a marker, for instance, on a person that's inside a ship, normally you would need to have all these zones all the way down to get him. In this case it's just that marker, and it just gives you the zone position of the biggest zone that's still streamed in, so the planet, or the root zone if he's just in space somewhere.
I'm picturing like when you're playing golf and you pick up your ball and you put a quarter down in its place, so that you know where that thing's supposed to be. The ball isn't actually there anymore, but you know where it's going to be. And when it comes time for you to actually need to interact with it, it's going to know where.
Kind of. And it's also got the advantage that we can do things like, because it's so far away, the position doesn't need to update constantly, because you can't tell if it's 10 centimeters out. So it can update every 30 frames or every 60 frames or whatever, which also obviously increases performance ever so slightly.
Yeah, and we don't need to get the messages from the server as often, so it's not affected. And then once you get in range, once it gets fully bound, everything else has been unbound, so the network performance and stuff. And then obviously there's stuff I'm not as involved in, so the radar scanning stuff has got to be able to work when the stuff it's scanning doesn't exist on your client. So a lot of the game code was changed from everything happens on the client to the client has to ask the server for some results, or it has to push requests to the server for things that happen.
Well, even the likes of when Clive was mentioning how we decide if something streams out is based on how big it is on your screen. So if there's a Gladius that's 20 kilometers away, it's going to be one pixel on your screen. It can probably stream out, or if it's going to be less than a pixel it can stream out. There's some adjustments made to that for the likes of the radar, so the Gladius will be made to think it's bigger, because it needs to stay around, because although you can't see it, your radar can see it. So there needs to be some kind of adjustment there, so you're still able to get the up-to-date and the high-frequency updates to see where it is.
TL;DRSerialized variables are one of several interconnected technologies OCS needs, carrying the data the server tells the client it needs for things like markers so gameplay can use it almost seamlessly.
Yeah, I think one of the complexities of OCS is there's a whole interconnected bunch of technologies that need to work. So there's serialized variables that need to be there, there's the actual bind culling itself, there's... I've lost my train of thought now where I was going with this. Other technologies as well that only exist for it to work. The serialized variables are kind of the thing that almost, I don't want to say glues it all together because it's a bit of a clichéd phrase, but for the likes of the marker, you'll have serialized variables that have all the data you need in it. The server will be able to tell you the data you need to have in those variables, how often you need to get it, and it will do it almost seamlessly from a gameplay point of view. So we've just got the data we need to use.
TL;DRYes, it is daunting: it touches many teams, and making the roughly hundred components on the actor thread safe creates a deep dependency graph across audio, AI and other teams.
Yeah, I think the amount of teams it touches, even... One of the changes we've been having to make for the async spawning stuff: all the components of the game have had to be made thread safe so they can be spawned off the main thread. For the actor, that means all the actor's components first need to be made thread safe. So you've got a bunch of components that need to be done by the audio team, a bunch that need to be done by AI, a bunch that need to be done by all these different people, and some components depend on other ones being done first. So I can't convert, for instance, the local player component until the particle stuff was converted. So we've got this big dependency graph that sometimes goes several levels deep before you can actually get to it. And there's just an awful lot of stuff. There's a hundred-odd components on the actor now.
Yeah, so our game runs off an entity component system. So the entity is like an actor. It has an actor component, it has a local player component, it's got an audio component, it's got a variety of different things. There'll be ones for animation and ones for effects and all this plethora of different things, which will be shared between different entities as well, so there'll be other entities that also have animation components and stuff. But yeah, everything had to be able to be spawned.
I mean, this has been an ongoing thing for most of my last quarter, so Q2 was spent doing this for the actor. A big, big chunk of that was we had a scripting language called Lua that we use extensively in the game, and it basically had to go because it's not particularly thread safe. There was no good way to make it safe for spawning things asynchronously, so it was a case of going through... for the actor there were an awful lot of Lua files, and there was an awful lot of stuff that had just been around since this engine was created. It has evolved over the course of the engine. And yeah, it was just a lot of tidying up and cleaning up and moving things around. And again, it was by multiple departments as well.
TL;DRSo far it speeds up loading the game for developers, makes the much bigger map loadable on machines with less memory, and lets more game clients run on a single test machine.
Current improvements that we've already seen? From a development point of view, it takes me less time to load the game, which is nice. So if I'm wanting to test very simple stuff, the PU takes less time for me to load because it doesn't have to load all the stuff that's out of range.
It actually makes it possible to load the game as well, because the map is so much bigger now. Not everyone here has quite large memory machines, so some of them can't actually load the full map because it's that big. So they need bind culling on. Well, we've got two maps. We've got a cut-down map and the full map. They either have to work on the cut-down map, or if they want to load the full map, they have to turn bind culling on to be able to actually get it in, working on their mission. It's that much bigger.
The reduced memory is helping us as well. I mentioned earlier about filling a server up, which is effectively using one tester's time across multiple machines. The less memory the game is using, the more game clients we can put on a single machine.
I think mostly it's going to affect RAM, that's where you're going to get the biggest saving. It should, I'm being a bit more cautious than Chris, it should improve things on CPU; it probably won't have much effect on GPU.
I think from my experience over the last week or two, there will be CPU improvements probably, but we're also ramming a lot more stuff in. So while there may be some gains, we've also got a significantly larger amount of AI.
TL;DRQA says the only drawback is testing time, while Gordon adds that programmers can no longer assume every entity exists, so features must be designed with streaming in and out in mind.
You mentioned earlier some of the challenges involved, like the scanning, having to leave the markers when something is culled so that people can still see it.
Yeah, I don't want to generalise, but most gameplay programmers are quite happy to just assume everything exists all the time. I'd be very happy if I just had every single entity on the client at the same time, because it'd make my job a lot easier, whereas now, I've got to make sure every entity I'm accessing exists, and if it stops existing while I'm talking to it, what do I do? Lots and lots of more technical challenges. It just requires you to think about things, not that you don't think about things, but you've got to be very aware of the implications that a lot of stuff might stream out. If I'm designing this feature, will it work when not everyone's here? Do I need to move it to exist on the server rather than the client? And tweak things to make them work.
I'd say any bug that you spot, report it to the Issue Council anyway and allow us to sort it out. Probably the kind of things you're going to see are perhaps missing entities, or stalls as you approach planets or derelicts or whatever. What else might you see?
Some kind of popping as things appear. The likes of teleporting actors, that's one that's kind of floating around a little bit, so just seeing guys jump around the place. Occasionally we see issues with that.
TL;DRThe speakers stress that it took about three years, is a company-wide effort touching every part of the game, will keep being improved, and will cause edge-case breakage, so testers should submit crashes.
It's been, as you said earlier, three years since we started work on various parts of it. Although not everyone's been working on it flat out for those three years, it's taken a massive amount of time, a lot of changes, tons of people. It's not down to just us in this room or our teams in particular. It's a company-wide effort. It's been a long road and it's hopefully worth it when we finally get there.
Yeah, I think just to reiterate that it's touched every single element of the game, and it is still currently touching every single element of the game. Like I said, it's a collection of features, but there's never been a feature where there have been as many emails that have come out to absolutely everyone going, is this going to break anything for you? It's needed everyone to pitch in. I see a whole lot more 'BCC everyone' on OCS threads than on just about anything else. And I think as well, as Clive said, it's not been a short road. Before even Object Container Streaming, there had to be object containers themselves, and that started as I was starting, so that's three and a half years ago. So that was kind of the very beginning of it, and this has obviously been on the plan since then, and it's just taken three and a half years of tech and improvements.
No, this is the kind of thing that's going to be ongoing forever, probably. There's always going to be things to improve and more stuff that is added to it. The server streaming is obviously the next major step, and then there's stuff beyond that as well.
Just to reiterate that it touches everything, and from the QA standpoint... everywhere. Yeah, it touches everything. There's going to be a lot of edge cases that, with our department, as cool as everyone is, there's only so many of us. There's only so many Evocati, there's only so many PTU testers as well, and there are a lot of things that are going to break, I think. Please submit to the Issue Council and always submit your crashes.
Yeah, so I'm a gameplay programmer and I'm on the FPS team in Wilmslow. So most of the stuff I do is player traversal. So I do a lot of stuff to do with ladders and cover and vaulting and jumping and all that type of thing.
TL;DRGordon says it was the first showing of the long-in-development cover system alongside FPS work on movement and weapon feel, and praises DataForge as a validated central database replacing hand-edited XML.
Yeah, so that was the first time, I think, the cover system's been seen ever. It's been something that's been in development for a long, long while. I've been working on it probably since back in November, give or take. So we've always been working on that, iterating on it. This is the first time it's really been shown off. So there's the cover system itself, and there's the general FPS stuff we've been working on, because it obviously had a lot of work. We've done loads of stuff on player movement and weapon feel and just all the core features of the FPS. And obviously you saw DataForge as well. It probably doesn't get shown off quite as much. I know Ash, the guy that wrote it, said cheers to me this morning, because that's the first time it's ever been given a shout-out.
We showed them DataForge twice. This was the second time we showed DataForge. We did a segment with Mark Abbott a couple of months ago where we showed it off. It's not one of the sexier features in game development, but it's such an important tool that we waited so long for. And Ashley Canning, right? Ashley Canning is a legend as far as I'm concerned.
As far as game development, one of the big aspects is games are basically a big database of values. It's lots and lots of numbers that you need for lots of different things. So having DataForge is just — it's a centralized repository where we can put everything, and it stores it well and it's all validated, and it checks against all, like if you've got a bunch of audio clips, it checks all the assets exist every time you try and export. Obviously, there's the tool itself with the live edit features. It's so good to use and so streamlined, as opposed to being lots of XML files everywhere that you've got to edit by hand. It's just glorious.
TL;DRWeapon raising already works in zero G, but the cover system is turned off there and will be gravity-only at first, though peeking around cover in zero G could be investigated later.
Yes, so like I said in the video, the cover system is based on the weapon raising stuff. That's the system that Ivo, one of the animation programmers in Germany, did. So that's already in, I think. It's already in the live universe. So that already works in zero G. So if you float up to something, your weapon's going to raise out of the way. The cover system itself doesn't work in zero G. It's just turned off in zero G at the moment. I could see there could probably be scope for it getting added in. At the moment, it doesn't hugely make a lot of sense. If you're floating around, you can be any orientation, you could be upside down, you could be face up to a wall, whatever. It doesn't really make sense to lean so much, but I could certainly see in the future we could have some kind of peek around cover, peek around walls, while in zero G. It would certainly be something that would be interesting to investigate. But certainly for the first iteration anyway, it's gravity only.
TL;DRStar Marine has had a lot of work since being pushed back, with new movement, weapon feel and animation refinements, and a recent team playtest felt like a cohesive, proper game.
So yeah, Star Marine has not really been talked about in like a year, I think. I think we were quite close to releasing it and then it got pushed back a little bit. We've done a lot, a lot of work on it since. Like I said, there's lots of new movement stuff, there's lots of weapon-feel stuff, lots of animation refinements, lots of stuff to just make the game feel a lot better and play a lot better. We've not long done a playtest over here, actually. It wasn't quite studio-wide; it was a team-wide playtest where a bunch of us jumped into the map and had a, I think it was 4v4, I'm not quite sure. And it's come on so far and it feels like a proper game now. It actually feels cohesive and something you'd actually want to play. It needed work, it needed refinement. But yeah, there's so much stuff being done in it. It's a smallish team that's working on it, but it is very dedicated to getting it feeling good, getting it playing well. Like I said in the video, obviously it's Todd in Germany that's overseeing it, and John over here. And they're really going to proper lengths to get it feeling right. There's loads of new features as well. I mean, simple stuff like all the reload animations have been redone quite recently. I think the last couple of weeks they went in. And it makes them feel much more — I mean, the whole thing about Star Citizen is it's as realistic as you can get, and they feel weighty now. They feel like a proper reload. And lots of other stuff, like selecting and deselecting weapons feels better. We've got crouch improvements, prone improvements and jumping improvements. And the jump was something that for a long while felt a bit pants. It always felt a little bit off, and so we got the opportunity to go over it, and all the animations got redone from the mocap. Dan over here spent a long time working those animations with Uzi as well, properly working them out, and I did the back code to glue them all in, and it feels a lot more natural.
Not yet, no. I know it's been mentioned, along with — at the moment, if you've got multiple people on the ladder, I think they climb through each other. I know there's been a lot of talk of either climbing over people or pulling people off ladders as well, but it's certainly something that's on our radar. I don't know how high the priority of it is, but again, it's something that would be quite cool to investigate, and quite cool to be able to pull your gun out and start sniping from halfway up a ladder. I know as well ladders were mentioned for zero G, because at the moment also I think they don't work in zero G. That will be something that will need to be resolved, and there will be some R&D going into that, probably.
Again, not at the moment, simply because of the pose: when you're prone, you're so low down, and for the weapon to raise up it would need to come out like this. Yeah, I mean, you can already do the rolls in prone, so I could see you being able to do something like rolling onto your side and peeking out. But again, it's not a feature we're currently working on, but it's something that'd be quite cool to R&D. Prone itself also needs a little bit of work. It's probably one of the glitchier features of the game just now. I know it's on the schedule to get looked at again. I know there's been some bugs fixed for it, so certainly that will be getting some love at some point as well.
I hadn't actually thought of that. There were non-cover leans at one point. They looked a little bit naff. You pressed the key and you just went... I know there's plans for sidling, which is like if you've got a narrow corridor, you put your hands up against it and sidle along. So I could see something similar for that: you go up against the wall and just look around or peek around.
Good question. I'm not sure; that would need to go to design. I'm not sure if they would be blocked or if they'd try and crawl over each other. Yeah, that would be a design question. That would normally be something that would be fielded by Todd or probably John.