Star Citizen: Happy Hour w/ Sean Tracy

Performance is one of those things that you've got to claw back every millisecond you can. It's everywhere and everyone has to pitch in on it. A big thing is on the asset front. I keep saying for a long time, artists have been able to kind of run renegade in the engine. Basically, nobody watching what they were doing. And then all of a sudden, we come in behind. We're like, why are we running at 10 frames a second? And then you actually open up the ship and you take a look, and the polycount's over the top. Or there's invisible meshes behind walls. There's so many times that we've come in and then found somebody was just going off the reservation, so to speak. But I'm pretty proud of the team for being able to get all that stuff together.

Actually, a pretty fun thing that got done for 2.6, and it's not something that I'm necessarily proud of, but I'm proud we're finally here, is that we're zeroed out on all the asset errors. When the assets get sent through the build process, we output a whole ton of errors that happen with them. There's things like, hey, your LOD 3 is actually higher poly than your LOD 0. You might not want to do that. Things like this. To give you an idea of where things were at about two years ago, and Tyler, you might remember this too, he remembered me coming into the studio and going... This is live, right? So I probably shouldn't say exactly what I had said, but it was a pretty exciting day. We were sitting around 16,000 of these errors. And we had to bug them all up.

I do recall this. They're like, okay, so all these errors, we can do them in chunks, right? So we can do them by theme or whatever. But there were still hundreds and hundreds of Jiras that had to be written, and it was a very tedious process. And there's a name of a tool in profile builds.

Error parser. I've made that all, well, actually, Paul helped me make that all automated as well. And we put that into a big report for QA now. It's on a super nice Excel sheet too.

Is it much smaller now?

What, the error count? Well, it's zero for 2.6, which is amazing. To go from 16,000 errors down to zero across all the assets is a pretty nice thing. And again, it's kind of funny to say because it's like, yeah, we were shipping out the stuff and there was asset errors all over it. But again, you never really ship in a perfect state.

Well, not in a project like this where we're releasing individual modules as they come online.

Yeah, well, I say it all the time. We're developing stuff faster than we can test it, is the thing. The reality of the situation is that there's new features, new assets, new models, new characters, whatever, all coming online much, much faster than we can test it. And that way, super happy to have the Evocati, the PTU testing, because that's when we get the real good visibility. It's a huge pool of users that are less judgmental, let's say, than the public would be.

Well, to modify that a little bit: every Star Citizen at every stage, whether it's the Evocati or the PTU or the Live Alpha, is a testing environment.

Yes, of course. Everybody is helping us test. And that's why it's still in Alpha. We're still making features like crazy.

Yeah, definitely. So anti-aliasing is one of those image quality things that is a graphics programmer's nightmare because there's so many different versions of it. A lot of people have developed different algorithms for it. Within Lumberyard or even legacy CryEngine, there was 10 different anti-aliasing modes. And in most cases, most games just pick one or two. But there's so many differences. Some anti-aliasing modes like SMAA, which is the morphological anti-aliasing. TSAA is another one. FXAA is another one. To kind of explain the differences between them: some of these are temporal, meaning it takes an amalgamation of frames before, so sometimes just one frame before or two frames before. But when you're running at a low frame rate, you end up getting a lot of ghosting, right? So a lot of people will blame that on motion blur, but sometimes that's actually coming from anti-aliasing.

So anyways, for Star Citizen, we actually keep this pretty low. We don't crank up the anti-aliasing by default. One is that the performance cost is pretty huge. And really, the future of anti-aliasing, and I've always thought this, is supersampling, which is you render behind the scenes at much, much higher res, and then you mix it down for whatever your screen is. I'm no graphics programmer, though, so Ali Brown might hear this and just freak out and be like, what is Sean saying? We can't actually render up at 4K and then bring that back down in real time. But I have seen it before, so that could be a nice feature.

To go a little more deep into it: once the shader development is done, that's when you kind of come in behind and deal with all your anti-aliasing. We used to have one within the CryEngine that was MSAA, and this was one that was developed in Frankfurt by a gentleman named Tiago Sousa, who worked a lot with NVIDIA directly. But the problem with that is that every time you're updating your shaders or every time you implement a new one, you've actually got to deal with MSAA on each of them. Again, it's kind of one of those polish things that we put a little closer to the end, when we're not rewriting or adding a whole ton more shaders.

But we want to give a lot of anti-aliasing modes, because that's the other part of it: every player is a little bit different. Some players really like that the diagonals are all really nice, but they're okay with no shader anti-aliasing, for example. So you get this mode. But if you want shader anti-aliasing and you're not so stressed out about the diagonals, then you're fine. You use a different mode. So it's a deep question and it becomes highly subjective.

There's some awesome stuff going on behind the scenes for delta patching. Of course, we don't have the most sophisticated patcher by any stretch of the imagination right now. It's just a huge amount of data that we've got to download and get out there. Again, I'm pretty sure this was all discussed long before I was even at CIG, because that patcher was in shape when a bunch of us arrived from Crytek. So I'm pretty sure the idea was, hey, let's just make sure we can get some stuff out to the community. Not really considering what the build size might be within two years, or maybe expecting that the patching would have changed.

So we are absolutely looking at this. This is happening in Frankfurt, with our DevOps guys in Austin as well. It's going to make a pretty big paradigm shift internally to us first, for developers. So to kind of pull that back, it's really important that we get you guys a delta patcher, but before that, if we change the way the patching works, we need that to work for us internally as well. Because if you think about it, we're sending those 40 to 60 gig builds all over the world for our developers. So first, it's got to work within the development mode. So we've already got a build online with it, which is really cool. So we've been testing it, finding out how it works from the developer standpoint. And it is a pretty fundamental change, because as Lumberyard is broken up into pack files, we have this idea of doing a little more uber of a pack file, and I don't really want to go into deep details on it yet, because a lot of the times I'll go into a detail and then we'll change it that next week.

And just as a quick side note, for those of you that are really interested in this topic, you should be able to find on our YouTube channel, maybe three months ago, a studio update with Mike Pickett in the Austin office, and he talks about a lot of cool information about the delta patching.

Yeah, and it's not that we didn't care about it or that it wasn't a big deal, but there was other things that were even higher priority to get worked out. Even things like just being able to kick up our servers. And you have to understand that DevOps is not a massive department, right? Within DevOps, are we talking like five or six people? Is it any bigger than that, Tyler?

They've got a handful more people, and I think they're hiring now.

Totally agree with you. I've been saying this all along, that we have a layer missing right now within Star Citizen. And it's going to become more and more apparent once you guys are flying around on the planets and have kind of a mix of vehicles and planets. So the layer that's missing, to me, is the layer between ships and infantry. One of the problems right now is you can sit there and camp a landing pad pretty easily, or any entrance that you want. As long as your ship can look at it, your infantry is screwed. They have no chance whatsoever. And some people say, okay, grab a rocket launcher. Okay, well, how many missiles have you just been shooting at a couple of these ships? One rocket's not going to do much. So what I've always thought is that you've got to have some sort of powered armor that can carry ship-sized weapons. Now, I don't even know what our max size is right now. Is it 15 or 14? It depends if it's a capital ship weapon or... Anyways, it should carry some of the smaller-sized ship weapons so that at least it can make the guy that's sitting there camping think twice and maybe push him out of the way. I always thought that's a layer that's missing right now. So we'd absolutely like to do that. And it makes sense. We've talked about the Titan, right?

It has been mentioned.

Well, the Titan's coming. So that's on its way too. But when I say coming, I mean it's going to hit the game at some point. It will be in a future release at some point.

Mega map is just a funny way to term it, I guess, because actually there's nothing really mega about it. It's really small. It's a tiny, tiny map. And the beauty of it is that it does every other map. So the meaning of that is that the engine, Lumberyard, has a concept of levels, right? And some people call these stages or however you want to call it. But basically there's a whole preload section that happens, and there's a concept of: this level exists in a space. It's got four kilometers by four kilometers or however big you want to call it. But there is a hard concept of a load of an area. Basically what we've done is just got rid of that entire concept completely, which is pretty fundamental within the engine. And instead we load one map, and actually it's really small. And then what we load is objects within it and around you. So a mega map is actually just a singular map that can support every other map.

The other cool part about that is we can shunt from single player to multiplayer and back even within that mega map. So you actually exist in one map. We load everything around you. And the nice part about that is, again, there is no offline loading screen that you have to go to. So if you want to go back to your hangar from Arena Commander, it's an instantaneous thing. We actually just stream the objects around you. And again, this required us to work a ton on the streaming that the engine does. When you have a concept of level, you can just preload a whole bunch of stuff and then stream in just the things that you need. But now, because we don't even have that concept, we have to get rid of all that preload that happened before and stream everything as much as possible. And that's something that we work on a lot. So I don't know if that really explained it well. Hopefully some people understand what mega map means. It's really just: we load one map at the start of the game, and we never load another map. We just load things around you, is the idea.

It's a good question. It's more of a design question. It can. And actually, it's something that's not super clear, I think, to the players even right now: the amount of ammo you have affects it, and the different weapons you have have different masses. The problem is that the relative mass difference is so minor that I don't think you really feel the difference. If you put a large-size weapon on one side and then a small weapon on the other side, your handling has actually changed marginally. But it's not as obvious, nor is it as readable as I'd like it for the players. So just on that note, cargo is just like every other item on the ship. It'll be an item that's attached into it. And if it has mass, it has mass and will affect the overall handling of that ship.

So it depends what you're shipping.

Yeah, it depends what you're shipping, it depends what ship you've got, and it actually really just depends how far our designers want to go in terms of how much that is going to change your handling. But the fact is that mass does exist in the ship, it does exist at a high-level concept on the ship, so it will change.

I think we've mentioned before that if you're trying to escape, it's possible you dump your cargo for an extra boost.

Yeah, and again, if you dropped off all the weapons and thrusters and things off the ship, I'm not sure that they'd even give you that huge of a difference right now. But again, if the designers want to put the numbers in so that it happens that way, absolutely, that'll happen. But the technology is absolutely there.

And the calculations are already being done.

They're already being done, yeah. It just becomes another component to the ship.

If I'm not mistaken, we are attempting to have one out very, very soon. Basically, if you haven't been following the This Week in Star Citizen posts on the website or ATV, we're at the tail end of our big beginning-of-the-year summits. Right now, we've got bigwigs from all the different studios meeting together while we plan out that schedule. So the reason you haven't had that schedule is because we are in the process of making it right now.

And HR decided that the best thing to do during our really important roadmapping time is that we have to do all our employee reviews at the exact same time. So all the directors that are absolutely required to make these roadmaps are doing employee reviews, which are very important, and they help our employees, and they push our people forward in the best way possible. But man, just do it next month. It would have been easier.

Mega map is exactly that tech, so I thought I'd tie that together. With the advent of mega map, that actually makes it a lot easier for us to do this. With that said, consider the amount of players we have, and then consider the landing location. So how do you put, conceptually, what could be a thousand people with their hangars into that level, so that when their hangar door opens, they've got a spot in there? We're coming up with some little hand-wavy ways, a little bit, so there's not going to be 10,000 hangars on one landing zone. You always come out of the same sort of area. But we're already taking a look at that, so that'll be a big thing. It'll be fun. It'll be nice to just actually see that damn door open and actually have a level there.

It's very common in Star Citizen's development. You push for the absolute realism and then you bring it back to what works for gameplay. You develop to make things as real and as awesome as possible, and then you have to tweak for gameplay reasons.

Yeah, exactly. And I say that to the guys all the time, that there's reality and then there's games, of course. And if you want reality, go outside, sit on the bus bench and come back and tell me how fun it was.

Nothing official, but I can tell you what I want. Sean wants the players to create their own tracks. I still think it'd be a lot cooler that the players actually create their own racetracks. If you think about it, these planets have so much area. So I can conceive that you can make your own Cannonball Run if you want to do something crazy.

There's going to be Dragonfly races, there's going to be Argo races.

Yeah. And maybe to explain what I had always envisioned with it: the player has a number of items, right? Call them beacons, call them gates, call them what you want. But basically they place them all down. And then, if they have buddies, they can place a start point. Maybe you've got just an area around that start point. And whether it's Greycats or Dragonflies or Hornets or whatever, wherever you put those beacons is where they've got to get to. And I would love that the players just set those up too. And then you can take that as far as you want to take it, right? Which is, okay, can you save that thing? Is it persisting there? Maybe it persists as long as that player's in the environment. How crazy do you want to get with that? Do you put tournaments in? Do you put votes for the racetracks? I don't know. But I always thought it'd be cool that the players would set it up. So I don't know of anything official with that. But I would say if we do do it, that's probably how it's going to go.

In many cases, we prefer it when the citizens set their own stuff up. Like the Star Citizen Combat League, that just started up like that. I'm talking to those guys now. We give them a couple of tips and pointers and stuff, but these things always work best when they're 100% community.

No, I think that's awesome. And the same thing that I've started seeing as well: we've got guys that are making videos now, we've got guys that are making machinima-type stuff. It's awesome. I love that. And I think players can set up their own gameplay better than a designer can. And of course, there's things that, yes, we want you to experience, and we're going to give you a nice long single-player campaign, scripted sort of stuff, unscripted sort of stuff. But in the end, there'll be emergence. There'll be things that are fun to do in the game that the players can set up for themselves. And that's great. And I think that's a good way to go about it.

I want to explain what Item 2.0 is because, again, this comes up so much and it sounds so nebulous. Like, oh, Item 2.0 is new items. What happened within the engine was that, again, with the level concept, we also had an entity concept, so that you place a bunch of objects in the world, and those are entities. Those entities have a very specific set of properties, and this is the animated object entity. So if you're going to animate an object, this is the object you use. If you're going to put a character down, that's an AI human entity. If it's a water volume, well, there you go. You put a water volume entity in. What Item 2.0 is, is it pulls us away from these preset kinds of entities, and it moves all those properties into a component system. So what this really means is that everything becomes an item. But then there's features that each of these items can share. A good example is a character item is very similar now to a ship item, because they can carry armor values. They can carry item ports. They can carry weapons. They can carry all their damage modification and ammo modification stuff. So we can put those components onto this kind of, let's call it a global entity, or let's call it a generic-type entity. So that way, when we get some pretty complex components, we can actually just plug them in and not worry about, okay, well, we need to be able to do that on that entity, we need to be able to do that on this entity. It's actually just a matter of making that one component, and those can be added to a whole huge set of different items.

So I don't know, maybe that went a little too long. It didn't really explain the item to another thing, but it's a componentized entity system, which is what Lumberyard has. And it's a super important thing to have. And it's funny because a lot of engines already have this. And when we had started with Crytek, it was never really set up as that. And even within Crytek, we had tried to do that a long time ago, but it's very, very difficult.