Graphics engineers cover progress on the Gen12 renderer and its Vulkan backend, CPU versus GPU bottlenecks, and mesh shaders. They also discuss graphics settings, borderless windows, temporal AA, HDR, motion blur, DLSS versus FSR, night vision, clouds, mirrors and 60 fps.
TL;DRVulkan, the back end of the new Gen12 renderer, lags slightly behind Gen12 and is roughly 60–75% complete; once Gen12 is done they expect Vulkan to follow soon after.
Vulkan is obviously going to be the back end of our new renderer, which we've called Gen12. And the Vulkan side of it is going pretty well. But ultimately, it's limited by our progress on the Gen12 renderer. Until that's complete, Vulkan can't be complete. So at the moment, as we've been developing the Gen12 renderer, Vulkan's always been playing catch-up, basically. As soon as something works in the new renderer, we look at Vulkan, make sure it works, and then bring it back up to speed. So it just lags ever so slightly behind. So in terms of features for the Vulkan renderer, we're probably about 75%, something like that, of that being completed. But probably slightly less when you start factoring in performance, because Vulkan will perform slightly differently to DirectX, which we were using in the past. Probably a naive first version we run internally might not be quite as fast, but obviously we don't want to ship that to you. We want to ship the faster version. So probably somewhere between 60% and 75%, depending on how much performance work there is. A lot of that has been done. Like I said, it's playing catch-up to Gen12, and Gen12 is progressing very well now. It's not far off, and once we finish the Gen12 aspect, we'll throw a bunch of bodies on the Vulkan side, and that will, I think, very quickly [catch up]. So we're not expecting a big difference between the Gen12 release and then the Vulkan release very soon after.
TL;DRGen12 is rolling out gradually in each patch rather than all at once, with opaque geometry and shadows nearly done; performance gains vary by machine, and the Vulkan switch will be a binary change.
No, and it would be lovely if we could do that, but the continuous live release means that to do that, we would have to keep both sets of code in parallel for like a whole year, and it's too painful to maintain both when we have constant features we're working on. So as we finish something, we leave it in an experimental stage for a while where we internally turn it on for the graphics programmers, and then once we're confident, we turn it on for the rest of the studio, and then the next patch that happens to come out will benefit from that. I think, as you already mentioned, all the post effects have already been converted. We've been focusing on getting the geometry done, which is the biggest part of the game. At the stage we are today in our internal stream, all of the opaque geometry is done, which is the vast majority of the game. And then the shadows are coming straight after that. They're basically done, just awaiting some testing. So really the big bits that are left are just to deal with the transparent geometry and then a few miscellaneous effects like the blackout effect and things like that, the final piece of polish. But yeah, the hard parts have been largely tackled. And transparency is still a problem. There's plenty of issues there. But the bulk of it is making very good progress and won't be too much longer.
So I just want to reiterate again before we move on that this is already coming out and releasing in parts, and sections of it are turning on. So backers are already seeing the work and results of some aspects of it. They'll see more aspects of it in 3.17. They'll see more aspects of it in 3.18.
So there will just be a time when it's like, okay, everything has now been converted over. But again, it won't be like a big boom, like you logged in on Thursday and the game ran one way and you logged in on Friday and now the game runs a different way. It's already in progress. It's already happening. It's already releasing. It's a little bit after a little bit after a little bit kind of thing.
Yeah. There's certain parts where there will be bigger differences than others. But the problem is, with a PC game, everyone's machine is different. So some people will be bound by that. Maybe they've got a really old GPU. Maybe they've got a four-core system versus an eight-core system. So everyone's going to have different performance characteristics. So there will be some unlucky person somewhere where Gen12 makes no difference to them, and there'll be someone else who sees their frame rate tripled. It's hard to give a general rule. But at each stage, we expect to see benefits. Probably the biggest ones might come towards the end, because the focus initially is going to be getting things working, getting things so they don't regress and get worse in terms of performance as far as the backer sees. But in terms of actual performance improvements, once it's all working, we'll be like, right, pile on, let's see what's actually not working as it should in terms of performance. And then the final switch to Vulkan is probably the biggest change. And that one is going to be a bit of a binary. You either use Vulkan or you're not. Everything else is going to be more gradual.
So that's quite a technical question. So yeah, each stage will get improvements in that regard for the people that have high core counts and low clock speed. Many parts of the renderer are already multi-threaded, and basically the Gen12 work is moving more and more to the multi-threaded stages, and less and less work ends up having to be done on the single core, what this person has correctly called our render thread, which will always remain on a single core. But in the final, final version of Vulkan, it will do almost no work. So everything else will be parallelized. But yeah, every single milestone will give benefits. So every release we expect from now until the job's 100% finished, you'll get an improvement.
TL;DRGen12 should remove much CPU work and make the game more GPU bound, but the game will likely always carry substantial CPU load as it keeps growing.
So the goal of Gen12 is to remove a large chunk of the CPU work. So that will naturally mean it's more GPU bound. But even if there was no CPU cost from the graphics team, from my department, the rest of the game is still big and complicated. It's a very ambitious game. So there's lots of work by all the other teams to reduce the CPU load, the engine team and the game team. It's an ongoing thing. It's constant work to improve the performance. Massive strides have been made, but at the same time, the game's growing. You see what might be minor improvements, but actually, if we hadn't been adding features, we'd probably get a much faster frame rate, but the game has to be completed. I'd imagine there will always be relatively intense CPU work on this game compared to a much more simple platform or something. Yeah, we want to move into the position where we are GPU bound from our team. That's what we want. And then we're the limiting factor. And then we can push the GPU harder for the people that have GPUs that can handle it, and we can scale it back and give people a good frame rate that don't have such powerful GPUs. So yeah, I think we will move to be more GPU intensive. But we have to be realistic that it's a complicated game. There'll always be a decent amount of CPU load.
It's an interesting challenge, because we are running a live environment in the middle of our attempt to make the game. So you build the universe out as it is, and then you do something that makes it less CPU intensive and puts more on the GPU, and it's great, but we don't just stay there. We then add more features to the game, more places to the game, more things that end up being CPU intensive. And then you have to work to bring that new stuff over. And then somebody comes in and adds more stuff to the game, which adds more CPU stuff. So it's this constant, never-ending challenge of development, because even after the day when we go live and we say 1.0, development of Star Citizen doesn't stop. We keep adding more planets. We keep adding more systems. So it is going to be a forever challenge for your team and all the performance teams to not just optimize for what is there today, but to continuously improve on everything that comes tomorrow.
I would hope eventually it becomes slightly easier. We've done a lot of building, a lot of core technology. As we get further along, you'd hope there wouldn't be so much core foundation, building the structures that allow us to build the universe. It'll hopefully be more game features, iterating on the fun, adding more mechanics, which should hopefully not be as intensive. You're either mining or you're hacking, you're either shooting or whatever the other mechanics are. They hopefully won't pile up forever. That's maybe different to the GPU side. We'll constantly add visual effects; whatever machines can handle, we'll add it. But hopefully on the game side, which is where we've struggled with CPU speed, that will ease slightly. The game's always going to be growing, and if the CPUs can handle it, there will be no reason not to add more complexity.
TL;DRMesh shaders interest the graphics team, but low player adoption means they can't be the only geometry path, so they're on hold until Gen12 ships.
Mesh shaders are something that certainly interests me greatly, that we would really love to look at. One thing we maybe have to temper that with is that the adoption rate currently isn't huge among the market for this new technology, I think. Rough numbers, maybe 30% or something of our player base. But that's obviously going to grow over time. And we also have to take into account that by the time you actually make the technology, the adoption rate would increase by the time you release it. But still, we have to be careful in what we use it for. If we made these new mesh shaders, which is a new way of building and rendering geometry, our core, only method of rendering geometry, we would lose half our player base, and that would be a huge storm. So we have to be wary that that's not a possibility. Now, you can use them for optimizations, which is one of the main benefits, but then you're giving an optimization to the highest-power GPUs that may not actually need it that much. So it's a difficult question. The ideal use is something that can improve the visuals, so the people with the more powerful, more modern GPUs get to see these new visuals, but somehow it doesn't impact the experience of the slightly less powerful GPUs still playing the game. We [don't] really want to cut people off. And I don't want this to be interpreted as holding back. It's more bang for buck. If we're not doing mesh shaders, we're doing something else. We're giving some of the benefits to as many people as we can. But at some point, the tide will turn, and the benefits will be clearer by using mesh shaders than by focusing elsewhere. So it's definitely something that's very interesting for us. And there's a few areas we've been discussing where we would potentially like to use them. But yeah, Gen12 is the focus. So that's on ice until Gen12 has shipped.
TL;DRGraphics team hasn't tested Linux with Vulkan; staff reportedly run the game via Proton with reasonable performance, Vulkan should speed that up, and the Easy Anti-Cheat issue awaits new SDKs.
Well, I'll preface it with: I generally don't have much involvement with Linux. That would be more the engine team and stuff. But I do know aspects. So in the graphics team internally, I don't run Linux, and I think most of our guys don't run Linux for the game. Gen12 is our focus, and Windows support. However, I know some of our employees have played the game and frequently play it via, I'm told, Lutris and Proton, the libraries they use on Linux, and that they get reasonable performance already. Now, switching to Vulkan will make that existing method of running on Linux much faster and should theoretically give you comparable or even better performance on Linux. I'm not sure if that's the only way we'll support Linux. I'm not sure if there's going to be a native client. That's a question for somebody else, for the engine team or CR. But I'm told there is a way, and that way is going to get faster on Linux. But we haven't tested it ourselves yet. The Vulkan support isn't at a point where you could directly compare A to B. Like I said, we're missing the transparent geometry at the moment, so it would be a bit unfair to compare. It might even run faster because it's missing parts of the game. Only once we've got Vulkan finished will we be able to do a fair comparison. And I'll just slip this in there, because people will ask. I'm told, and again, this is not my area, that there is an issue with Vulkan at the moment with Easy Anti-Cheat, which I believe Epic are looking into. And there are SDKs being released in the coming months, and when they are released, we'll be integrating them to try and resolve that issue. But again, that's outside my sphere of knowledge, I'm just dumping information. I knew there would be Linux questions.
TL;DRGraphics tech churn is manageable since most changes are straight replacements, but all existing assets must stay compatible, so old and new shaders ship together until artists migrate assets.
So I think every AAA studio with its own engine has the same challenge, really, of keeping up to date. They constantly have to make new post effects as new technology or new ideas come about in the industry. So that's not massively different for us compared to any other, I don't want to name them, but I'm sure you'll know the ones that aren't using big engines or part of multi-studio publishers. So I don't see that being massively different. We only have a couple of differences internally. One is we can't wipe the slate clean every couple of years when we release a game, or however frequently a big studio might. So we can't say, well, that was good, we did mostly good, but we didn't like that technology, so next time we're going to do a different one. We have to build the next version of whatever it is while we're running. So that does introduce some complexities. But overall, especially for the graphics team, it's not been a problem for us. Most things are straight replacements. You replace the old bloom effect with a new bloom effect. You replace the old shader with a new shader. It's not a major problem. I've been in a quick dev cycle at a previous company, and it's not that much different. I don't find it a major challenge. The one point that I would say is different, though, is the art pipeline and the artwork we have. We have to maintain everything that has been built. We can't say, oh, we've done this new technology, but these five spaceships don't work anymore. That's just not an option. Everything we make has to remain backwards compatible. And what we tend to do, say for the art pipeline, is we make a new shader that can do, I don't know, like the tint system for the spaceships so we can more easily roll out new skins. That system won't work in an old spaceship. So we'll ship two versions of the shader, the new one and the old one, and then when somebody gets the chance, they'll go back to that ship, assign the new shader, maybe fix up anything that needs fixing up, and then we're good again. We can't demand that they work at the same frequency as us, so we'll do the new technology and they will opt into it when they get round to it, which might be the next week, it might be a few months. And once we get close to around 80% adoption, we'd have to poke people with a stick to try and get over the hurdle so we can delete the old technology. And that's constantly happening. We have lots of different technologies that are in the process of being rolled out across the assets. So that's the biggest issue. But it's not really a technology issue. It's more of a data management issue with the size of the content of the game, which is very huge. I don't even know how many spaceships we have now, but it's big numbers.
TL;DRAli Brown says other engines pushing new tech is the normal ebb and flow, and as long as the engine and graphics team keep being developed, Star Citizen's visuals should keep up, though not instantly.
Yeah, I mean, it's a similar question to the previous one, in that every developer that is not using Unreal or any big engine is going to have that same question. And it's not really that different, to be honest, than the last 10, 15 years. Maybe since, say, Unreal 3 came around, when we started to see these big engines and also the publishers that have been joining more devs together. So I don't think this is anything new. Obviously, there's always going to be the big new shiny thing, and some company, not always Unreal, sometimes other devs, will come up with the new shiny thing, and everyone will learn from that, and they'll trial it, and they'll look at the success or failure of that engine, that game, and see if that technology survives the test of time. And if it does, other studios will naturally make their own version of that and absorb it into their pipeline. So I think that's just the natural ebb and flow of the industry. Somebody will push ahead in one area, and other people will eventually catch up. It's continuous. I don't think it's always one engine, though. There are things that the larger companies can push much harder on, like the tooling, because they are selling a product, their engine, to people that maybe are not even professional game devs sometimes. So they have to make the tooling very powerful and easy to use. It's less of a focus when you have professional devs that are very used to the engine. So yeah, I'm not going to pretend that we don't look at, say, the latest UE5 technology and say, that's pretty good. There's some really impressive stuff there. Nanite in particular is very interesting. But I'm not particularly worried, as long as we continue to develop our engine, as long as we continue to maintain and grow our graphics team and keep up with the latest technology. We will be there. It just doesn't mean that the day after Unreal implement Nanite, we're going to implement Nanite. And obviously, when the latest thing comes out, and we're not talking about Nanite here specifically, just in general, there is always the assumption that it solves all of life's problems. And then you wait six months or a year and you find out that it solves a subset of problems. It's not the solution to everything. There are many aspects to a full package, a full engine like we've got or Unreal.
I think so, yeah. I mean, our visuals are still cutting edge now. There's areas of the engine that we are extremely proud of that will match or surpass other engines, and there's areas that are not as good that we need to improve. So I don't see a major issue there. And for example, they picked Unreal, so I'll talk about that, but the Nanite tech, it improves the quality of geometry. That's super powerful in some areas. It's a benefit everywhere, but the main thing is natural geometry. So for planets and things like this, that could be a great benefit. But it doesn't do as much for some other types of geometry, internal spaces. And I think our quality on our interiors has been extremely high for a number of years and matches any other game I see. It's not that we don't want much better natural geometry. Yeah, we'll get there. But I'm not worried as long as we maintain the development, maintain the team. As long as you're not going to fire us tomorrow, we're all good.
TL;DRAli Brown doesn't think the engine hits a limit, saying graphics development is accelerating now that the renderer has been replaced, and assuming the team stays the same or grows, more visuals are expected.
So I don't think so. It's almost the opposite. First off, an engine isn't a single thing, obviously, that has a limit, where you hit it and then you stagnate or you replace it. It's many, many different subsystems, and subsystems of the subsystems. And these, like I've mentioned before, are being constantly iterated or replaced. And that's not a problem. That's not wasted. That's just the natural evolution of software. So in terms of whether we can maintain it, I think it's the opposite. I think we're accelerating quite quickly. I think the initial years, say for the graphics team, were finding our feet. We obviously inherited a code base, and learned that, and then brought more people on board to better understand and start to develop that code base. And more recently on the graphics side, we obviously replaced the renderer. So now it is 100% ours, and 100% our understanding of every nook and cranny of it. And once that's done, our rate is going to be much greater to be able to keep up with new technology. And that's the big motivator for Gen12. Performance was one, but the secondary one was to be able to adapt to new technology and create new features much more quickly. So yeah, I think, again assuming the team stays the same or grows in size, we've got plenty of capacity now to develop things much quicker than we have done in the past. And we need to, obviously, to keep up with the big engines. But yeah, I'm not concerned in that way. I think we're going to be pushing a lot more visuals in the coming years.
TL;DRGen12 is just a name, Vulkan was chosen as the better fit, and there are no plans for DX12 now, though the engine could support another API if needed.
There's no immediate reason for DX12. We started planning this renderer long before we'd necessarily settled 100% on Vulkan, and we just had to pick a name. DX12 has a numbering system. At the time, Vulkan and the other APIs didn't have a numbering system. Everyone understood what DX12 was in the industry. So we went with something that had the number 12 in it. It helps pin it down to this piece of tech. And if you call it the new renderer, then the next one is going to be the new, new renderer and the better renderer. So it's called Gen12. But in the end, we decided that Vulkan was the better choice for us because of the Linux aspect and a few other [things]; from a technology point of view, it made it slightly better for us. But it wasn't a big deal. We could have gone DX12. But at the moment, there's no pressing reason for us to implement DX12. We could. It wouldn't be a huge effort. Put a couple of guys on it for some number of months, and you would eventually pop that out. But unless we have a reason to believe there was some new technology on DX12, or some platform or something that required that, there is no reason for us to really go there at the moment. So for now, no plans for DirectX 12, but the engine is set up in a way that we very much could implement another API if we needed to. If the successor of Vulkan comes along, or DX13 is the thing that we need to make the world better, then we will do that.
TL;DRGraphics options haven't had much attention, but after Gen12 is done they plan a fresh look, with simpler basic settings, a more detailed advanced section, and only options that meaningfully change performance.
Yeah, I have to admit, we've not spent a great deal of effort there, as you'll probably see from the comment. But we've been holding off for a while. Because we were mostly CPU bound, and graphics options are generally there to squeeze out performance. We generally wanted the highest quality most of the time, and it's only to scale back, and mostly we would not be GPU bound. So there wasn't a huge motivation for scaling the engine back. All the focus is getting the CPU cost down, which is what the servers need. It's what the clients need. So that's been the focus. And then implementing Gen12 changes the way the renderer works, so it changes the way we would expose the options. So we're going to get Gen12 done, and then we're going to have a fresh look at these options. They will probably have quite a radical change. We're going to simplify them a lot, but then also we'll have an advanced section that will be much more complicated than we have now. At the moment, I think there's four different quality settings, but we're going to maybe change that up, maybe reduce that a bit. But then the memory management is something we're probably going to push much further. But yeah, we're definitely going to have a whole bunch of new options and make sure that the options we have give you some meaningful change. Some of the settings we have there, you can change things to high or low, reduce draw distance or increase it, but you'll find that the frame rate doesn't really change. And it's a pointless setting, so why have it? So we're going to set them based on what's needed to change the frame rate by a certain fraction. So high might be 20% slower than medium, and low might be 20% faster. That would be the motivation. And if it doesn't change the performance, we won't expose it.
TL;DRA borderless window mode exists but is limited to the main monitor's resolution; fixing multi-monitor spanning and resolution scaling is high priority after Gen12, with no exact time frame given.
So we already support a borderless window mode, but I gather, because they've talked about multi-monitor, they're probably meaning that I think it centers onto your main monitor at the moment and it locks to the resolution of your center monitor. So you can't drag it out to span two or three monitors and customize the resolution. Say if you've got three 4K monitors, maybe you don't want to run at 12K, unless you've got a monster GPU. So yeah, definitely. It's a bit of an embarrassment we haven't quite got to that yet. That's definitely something that's very high on our priority list to fix as soon as Gen12 is done. So resolution scaling, and then picking the size of the window and letting you move it around. Yeah, definitely.
And internally, we already have the resolution scaling, which is one aspect of it that people have been asking for as well. But that doesn't quite work with our UI yet, so there's just a few bugs there. And then changing the size of it shouldn't be a complicated issue. So yeah, I can't give an exact time frame, but we'll definitely be looking at that very soon.
I tend to play almost everything in windowed borderless because I need to switch over to Teams or email or something on another monitor very easily, because in the middle of the day, CR will ping me or something. I always have to keep an eye on the left screen while I'm playing on the middle. So windowed borderless makes that experience a lot easier for me.
There can be some quite technical differences between when you run full-screen borderless and exclusive full screen. It depends on lots and lots of factors. But there are potentially changes in performance. It depends on your system; either could be faster. And there's also changes in the latency as well. Generally, full-screen exclusive is less latency. But yeah, it's a pain when you're getting messaged by someone in Discord or something and you want to alt-tab out.
TL;DRA hard off for temporal AA is probably not possible because it would break shadows, occlusion and terrain blending, but they may look at reducing its strength or sharpening options and at the ghosting.
Ah, that's a thorny one. Yeah, so temporal AA, anti-aliasing, is one form of what we call a temporal accumulation system. It uses data from a previous frame to basically just make things cheaper. It opens the door to a whole host of new technologies. Anti-aliasing is one of them and the most obvious, but it actually is a major way we can reduce noise in all of our rendering. And also, it can help you introduce blending effects. You can blend between two things. For example, currently our planet terrain uses the temporal blending to blend between all the different ground textures. In terms of turning off temporal AA, if you turn it off, then you don't just turn off anti-aliasing. You turn off all the temporal blending system. So you'll get noisy shadows, noisy occlusion, no blending on the terrain. So yeah, it's not quite as simple as turning it off. This is an important reality of modern rendering, and you'll find that in many engines, like 70%, 80% of them, that is the case, because it gives you so much performance. It lets you spend it on everything else. And usually it's only quite bespoke, quite narrowly focused games that can afford to not use it and can blow the bandwidth on a different form of AA that is usually more expensive, like MSAA or something like that. So unfortunately, a hard off at the moment is probably not possible. We could potentially expose the option, but if we did that, it would probably have to come with a massive warning that what you're about to do is going to break the visuals. And what I don't want to see is pictures of our game looking awful and broken, and people posting, look how bad this game looks. But maybe there's a halfway house. Maybe we decrease the strength of it, but with a warning that some effects will become more noisy as a result of that. So you'd think that it's just going to make the image crisper or sharper, but actually, you'll be degrading it in other ways. So maybe not a hard off, but there might be some flexibility in there. We can have a look at that when we look at the graphics options to see if there's something that makes sense. It's a tricky one, though. It's an unfortunate reality. Nobody likes temporal AA in the company. Especially UI people hate it because of the potential [unclear: 'messaging laws']. You'll find everyone from VFX hates it because it smudges out their particle effects. It's not a nice thing. Nobody appreciates the alternative, but it's amazing what it can do. It is vastly superior to the alternative, so it's tricky. But maybe some people just want the image sharp, so having sharpening controls or tweaking that is OK. But the ghosting specifically, which this person mentioned, is tricky, because that is a product of using the data from the previous frame in an area of the screen where you don't have data from the previous frame. So if you turn the camera quickly or somebody runs in front of you, you don't know what was there previously, so you have to make it up. And basically what you're seeing when there's ghosting is sometimes actually the lack of temporal data. So that's a clue as to what you might get if you didn't have temporal data. We'll see what we can do, see if there's something that can help with that.
TL;DRNative HDR is coming: most of the pipeline already supports it, but HDR color correction needs art review and testing across formats, all planned after Gen12 with no date given.
Yes, absolutely. So we already support HDR through most of the pipeline, and all the rendering is HDR. Unfortunately, there are two things left to do to complete that. One is our color processing. So generally what happens is you produce your image in a high dynamic range, and then you crunch it down to a low dynamic range, which is the bit that HDR needs to modify. But then we color correct it, just like you would in a film. You adapt the colors. You try and produce a mood. And that process is built in low dynamic range. And it's an artistic process. We can't just turn it into HDR. It needs to go back to art. We need to review how they would like to do this color correction in HDR. It's a reasonably known problem. Other studios have tackled it. But that's just the piece we haven't got to. And then there is a minor bit of work in making sure that it works in all the different types of HDR. There are so many different formats and so many different brightnesses of screens and monitors. You have to do a bit of careful QA to make sure that you actually get good results. There's no point just testing on my fancy HDR screen and then having a normal consumer one not actually look good. So yeah, there's two little bits to finish off. The continuing theme here is after Gen12. We really just want to get Gen12 done. But yeah, that's quite high on the list. It initially was not as important because HDR monitors were relatively rare in the PC market compared to consoles, which you'd play on a TV, which tends to support HDR. But it's definitely become a lot more popular now.
Yes. And I'm slightly ashamed that we don't already support that. I know that our Digital Foundry friends have pushed hard for that. And yeah, clearly, camera motion blur is contentious, to say the least. Object motion blur has some public backing, far more than the camera motion blur, which is notoriously hated by a lot of people. Good for cinematics. Great for cinematics. Makes it look amazing. It's wonderful for cinematics. While you're in the fast-paced action, it's controversial. And I think there is a slider, I can't remember if it's exposed at the moment. We have the ability to tone it down. But yeah, most people were just like, off. I don't want that part of it. It doesn't add anything. But things like character animations and gun reloads and stuff can look a lot better with some per-object motion blur if it's at the right level, not too in-your-face.
TL;DRVRAM usage will steadily increase, but the minimum is expected to stay stable for quite some time, and Gen12 will show memory use in advanced graphics options so players can balance it themselves.
That's an interesting one. So we've had to be quite careful in the past with the amount of video memory used. We have backers that backed early on that don't have quite so powerful GPUs. They don't hold us back; we're just trying to make sure that we don't rule them out of the game. So we've been quite smart in how we've used the memory, and we make sure that they're still able to run the game. Obviously, the performance will not be as good on their machines. So yeah, I think with three to four gigs, you can run the game quite comfortably and get decent visuals. And I think around the four-gig region will stay running the game quite nicely for quite some time. I don't think there's a technology reason why the minimum needs to increase. But what will happen is we have all these different pools of memory: how much is used for textures, how much is used for geometry, and so on. And the core amount of memory you need to run everything without textures or without meshes is increasing over time. And eventually, out of your four gigs, if we end up with three gigs to do the bare basics, then your one gig left won't give you a lot of textures and a lot of geometry. You'll still be able to run the game, but eventually, yeah, you'll start getting diminishing returns. And there's elements of that already. When we added planets, gas clouds, these things all took up memory. I think the gas clouds budget is like 300 megs, 400 megs or something. So that has to come from somewhere. And if your machine doesn't have it, you're getting less of something to pay for that. Now, one thing we are going to do with Gen12: as a necessity of the way Vulkan works, the memory management is much more explicit. Basically, rather than DirectX or NVIDIA or Intel managing the memory for you, it's left largely to yourself as a developer. But as a result of that, we are much more strict with how we use the memory. And what we're going to do is what a few other games have done, which is expose that to you in the advanced graphics options so you can see exactly what memory you're using and balance that as you see fit. So if you want more texture memory, then you will sacrifice something else. That's your choice. And you can see how we're adapting, rather than it being hidden from you, which it slightly is at the moment. And the other thing that a lot of people maybe don't realize is the resolution you run the game at has a huge impact on the memory. So when I'm saying 4 gigs is OK, if you're trying to run the game at 8K or 4K widescreen or something, you are taking a huge amount of memory for the bare bones of just getting the screen rendered, without any geometry or textures in there. So you leave very little room for that. So if we expose it to you and show it to you, it'll be much more clear, and people that don't have as much memory will be able to make a smarter decision on how they want to use it, if they're interested. Obviously, we'll have automatic settings that do our best job. But yeah, should you want to delve into that, [you can]. So to answer the question, yes, VRAM usage will steadily increase. But I don't think our minimum will increase for a while. I think it will be quite stable for quite some time. Especially at the 4GB region, it's going to be OK. But yeah, we definitely want to be able to push to 8 more eventually. And we need to, because for example, the consoles have significantly more memory than 4GB, and we need to be beating them. We need to use it, but we just need to keep something in reserve for the 4GB people.
TL;DRAli Brown says heavy battles like XenoThreat are mostly CPU-bound; Gen12 will help but isn't the only part, and game and engine teams must keep optimizing.
I think a lot of that will be CPU issues, usually. I mean, we'd have to profile each case. But when we've looked at things like XenoThreat, it's nearly always a CPU cost. And generally, in a space battle like XenoThreat, in terms of polygon count, it can't be that high. I mean, you're in space. There's a limit to what you could see on screen. Even if half the screen is filled with space, that gives you an enormous amount of budget to spend on the bits that aren't space. So yeah, we do have some things like gas clouds, which are more expensive in space. But otherwise, it's not usually a GPU concern in space. It's CPU. Now, to beat the same drum again, Gen12 is one of the major parts of the solution for CPU performance, but it's not the only part. And when it comes to XenoThreat, that will be the game code. There will be a lot happening there. There'll be a lot of lasers firing, all sorts of things happening. The optimization there is largely going to come from the game teams, I think. Obviously, as we're removing as much as we can from the rendering side, the graphics, the CPU cost of that will free up more time for them. Ultimately, I think we'll get to a point where we'll get better, but then it'll be on their teams to take the mantle and push it further. So yeah, I think we can handle it from the graphics side. As we do Gen12, we'll definitely improve how easily we can handle it. But then the game team and the engine team will have to continue to optimize, which they're doing every day. There's constant emails, constant work being done on that side.
TL;DRAli Brown can't promise night vision since that's a design decision, but he describes how a more realistic implementation could work and says he would love to explore it.
Well, much in the same vein as you don't want to promise thousands of players to the backers, I can't promise game design features. I think various game designers are interested in it, but I will not speak on their behalf. But what I can do is talk about night vision implementations and how it's done, and leave it to the designers as to what that means in Star Citizen. So in terms of night vision, almost every game up until the last couple of years has done the same thing, which is make the screen go a bit green, change the colors a little bit, brighten it up a little bit, and job done. There's your night vision. So that's fine. We can do that, no problem. But recently, we've started to see people going a little bit further. I don't know if anyone's played Call of Duty: Modern Warfare, but they did a significantly better implementation. It's worth looking at. They really pushed the boat out a little bit further and tried to emulate exactly what night vision does, or a particular model of night vision.
Yeah, so now they better approximate what each color looks like when looked at under infrared. So they convert the colors that you would see. When art says this is going to be a red can, they do the maths and figure out what that actually looks like under an infrared camera. And the other thing they do is for things that have particularly non-standard, unexpected brightness in infrared, like a reflector or an infrared light or whatever. There's certain materials that will reflect very brightly. The artists have marked this up, so they have this data to give you a much more convincing visual. So yeah, that's definitely [unclear: 'in the payment one']. The one thing we have to be careful of is the amount of artwork we have means we can't just go and ask the artists to flag up every single material they've ever made. But we do already have various markup on materials so that we know the rough surfaces, the types of things they're made of. So we can retrospectively say, oh, all the ones that are made of metal, this is how they should respond. All the ones that are made of reflective strips, they respond like this. So there should be scope for us to do something like that. And that's definitely what I aspire to. I can't promise, because we're not doing that right now. Until design say today is the day you make the night vision mode, that is the day I can talk more freely about it. But yeah, it would definitely be an interesting thing I'd love to explore.
TL;DRCloud noise at near-horizontal angles comes from sampling along very long rays and randomizing samples, and a promising low-resolution pre-pass should improve it, with improvements coming but no release talk.
So the cloud technology is something our engine director has been pioneering for some time. So I want to be careful not to speak on his behalf. Many of the other things we've talked about have been my remit, but this is definitely his baby. But I do know a bit of how it's implemented, so I can speak a little bit on it. So the clouds are built from many different stages that go into that process. I can't remember, say five or six different stages, and each of them has been R&D'd in multiple different alternative solutions. And we're constantly finding new and improved ways to replace each section. And the noise in particular, at the horizontal angle mentioned, is because of the way that clouds work in our engine, and most others: you have an imaginary ray that flies through into the distance, and you take samples along that ray. So say I take 50 samples between here and the horizon, and I do some calculations to decide how much cloud cover there is at each of these points. Then I add them all up and I eventually come up with the final contribution of clouds. But if you imagine you're looking from orbit straight down, or from the ground straight up, the amount of cloud cover you see versus looking horizontally is hugely different. You can look through 100 kilometers of clouds versus one kilometer if you look straight up. So when you're running, say, 50 samples over 100 kilometers, there is a huge amount of error. And that would show as a stepping, blocky artifact. So that you don't see blockiness and stepping, we randomize where we sample along that ray, so it turns into noise. So we have to get into R&D solutions that will vastly improve that. There's already many mechanisms to try and minimize that. But one thing that's showing very good promise is we do an initial render, basically, of the clouds at low resolution. I think it's quarter or eighth resolution. Try and identify where the cloud is, where the noisy parts are, where most of the detail is. And then when it comes to doing the main tracing of the clouds, we put all the rays and the samples where we need them. So we adapt to say, oh, this part of the scene has a fringe here, a silver lining on the cloud. That looks really nice, but it's noisy, so we'll fire many more samples there. And this next part of the screen will be big and foggy. Nothing's happening. No point spending all of our GPU on that. So we cut back at that point. So it optimizes it at the same time as improving the quality in different areas of the screen. That is looking promising, like I say. But because it's the engine team managing that, we won't be talking about releases and stuff, but there are definite improvements coming on that side of things. And it's going to continue to be added on for some time.
TL;DRRay tracing is the only sensible way to get true mirror reflections, though simple cases like a single mirror could be done by re-rendering the scene, which the team has considered but handheld mirrors would be too costly.
We have plenty of shiny surfaces in the game, but I gather they mean full-on mirror reflection, really, really clean. For true mirror-like surfaces, ray tracing is the only sensible way to solve that problem. There's no real alternative. There are, however, much simpler solutions for bespoke cases. There are many games where you've seen a water reflection or a single mirror, a typical mirror, and they can be handled just by rendering the scene again from a different perspective, usually at lower quality or some kind of compromise to stop it halving the performance. And they're quite viable, and we would probably consider doing that, especially because it's quite simple. And for the mirror case, it's a nice, easy way to work. It does require just a little bit of constraint from the artists not to do like a hall of mirrors. You put a mirror in a bathroom, and when we've told art to test this in the past, they'll do it, and then behind the bathroom there'll be like a bungalow or something. And you're like, no, please put the mirror in a [sensible] location.
Yeah, I mean, you could, but you would just halve the performance, literally, of the game. And although you'd hope a handheld mirror would be smaller, some parts of the engine will not be able to scale down. So the number of pixels will be less, so that's great, you'll save on something. But there'll be certain aspects of the game that will not scale down. If it's just a tenth of the screen size, it doesn't mean it'll cost 10 times less. It might still cost like 50%. So who wants to take a 50% frame rate hit to have a handheld mirror? And if you've played any racing games which have rearview mirrors, you'll always see the lower quality they get in the rearview mirrors. Or they have such a bespoke engine to manage that, and that's a core feature. Unfortunately, we've got an extremely complicated game before we consider mirrors. But for small mirrors, you'd probably just render the scene flipped. That is something we've been considering. There hasn't been a really strong need for it, but there were some needs from some departments asking for it, which is all I'll say there. But when you get to the fully general case, like seeing your reflection in a spaceship's curved surface, ray tracing is the only real answer there.
TL;DRThe ground effect system could already support different kicked-up snow effects per surface type, but full snow weather is on the internal roadmap with no timeline and would be a bespoke effect.
So two parts to the answer. One is I don't know if the dust particles are literally the same particles in the two examples there. They might be the same. They might be different. The tech is there already for them to be different. We have quite a complicated ground effect system. It's not strictly speaking weather. It's the wind kicking up whatever is on the ground. So if there's snow on the ground and there's a lot of wind, it will kick the snow up. We already have the ability to do different effects per, we call it surface type, each type of material on the ground. So that, in effect, could be done now. It's more of an asset VFX type question for the art teams. However, to get to probably the real answer, it's not snow. It's snow being kicked up in a blizzard, basically. To get strong winds, to get full-on snow, full-on weather, that's something different. And it is on our internal roadmap. And I think a lot of people are quite excited to get onto that. But I don't have a timeline I can share on that at the moment. But yeah, clearly, when we implement full-on weather, that will be a bespoke effect. Snow does not look like dust. It has a very specific look. And you'd expect rain and all the other types of weather, hail or whatever. So yeah, it's on there. It will be coming one day, but not today.
TL;DRDLSS and FSR will be judged independently on their own merits, with implementation time the biggest factor and market share a real but not deciding one; neither is guaranteed to be implemented.
So yeah, those two technologies obviously aim for a similar goal, but in the way they're implemented, they're completely different. And they have quite different performance, different quality, and different suitability to the engine and the problem you're trying to solve. And also how easy they are to integrate is quite different. They fit into different areas of the pipeline, and both of these techniques introduce a few complications to the engine, particularly with the UI, because in most games, there's my game world, upscale it, slap my 2D UI on, it says how many bullets you've got or whatever, and then job done, that's the frame. But our UI is largely in the world. And also, we don't have one type of UI. We have so many types of UI. We have the options menu. But other than that, we have mobiGlas, which is almost like a traditional UI, but it's not. It's still in the world. You can still see behind it, people running up to you while using it. We have the holographic ship UI. We have the MFDs. I can't remember all the different names we have, but things that are in the world, Inner Thought they call it, when you're going to talk to someone and it comes up with the options. There are so many different types of UI, and none of these play nicely with DLSS or FSR. So there's some work for us to figure out there. But yeah, as they mentioned, the market share isn't equal. One is currently supporting every GPU, if they can afford the performance, and the other is on a much smaller percentage. I think 30%-ish might be the region we've been looking at for DLSS at the moment. So yeah, for me, it's not really a question of A versus B. They're going to be considered independently on their own merits. Can we implement one? How long will it take? How do we implement it? But I can't pretend that market share isn't a factor; it's definitely a factor. If we have something that is going to benefit 99% or 100% of people, we want to implement that over something that benefits 10% of people. On the other side, say we implemented one of them and it didn't give great results, we have no obligation to implement that. And I don't want to pretend that we, or any other company, haven't implemented technology that only benefits a subset of people. If we implement HDR, which we talked about earlier, if you don't have an HDR monitor, tough luck. Ray tracing: don't have an RTX card or the latest AMD, tough luck. And optimizations for one CPU brand or the other, or face over IP, these are many examples where it benefits a subset. Now, it's not great to be benefiting a subset, but we wouldn't have any of those technologies if we only tried to satisfy 100% of the people all the time. So it's more of an independent decision, bang for buck. I can see both of them being implemented. And the time to implement is probably the largest factor, to be honest.
TL;DRHe declines to say which is better, but says DLSS currently has a higher quality ceiling because it uses more input information, while FSR deliberately uses less, and either could change.
Well, I can't answer what is better or worse. I will leave that. That's technology owned by two different companies who are integrating it individually, so up to them. But what I will say is, DLSS has a distinct advantage in the way that it currently works, in that it is using more information that goes into it, so it has a better chance of giving a good result. FSR intentionally takes less information in, so there's a limitation to what it can actually achieve. So there is a fundamental limitation there. Now, either of them could change tomorrow. These are two independent companies making their own decisions on how this technology works. But at the moment, the upper limit is higher for DLSS because of the inputs it takes in. And that's a decision you just make. You have technology that can work different ways. We talked earlier about temporal solutions. If you use data from the previous frame, you have a much better chance of getting a good result, but you introduce a whole bunch of potential issues with ghosting. And you'll get one person that thinks that's a fair trade-off and other people that think that's not a good trade-off. So it's quite subjective what is better. But if you had a static camera, then having the extra information of DLSS gives you a chance of a better image. But I'll leave it to the public to decide what's actually better.
TL;DRHUD legibility is a known, non-trivial problem, and new shader effects like drop shadows and background blur are being brought into the UI pipeline, with a rough timeframe of weeks to months.
So yes, obviously the UI team would have their part to say on this as well, but we constantly hear from them, and actually spent a good time today talking about this issue with the UI team. It's not a trivial problem. We have things that are insanely bright in the game, like looking straight at the sun, and there's always a subjective nature as to whether that should be blinding. It is incredibly bright, and it is a factor in real life. But yeah, there are many times when it's not as legible as you'd want. So on our side, we've done quite a few different shader changes, which are mostly done, that are in the process of being brought into the UI pipeline to be used in the game. These are things like adding drop shadows underneath text. We have ways, like you probably noticed when you use the mobiGlas, it darkens the scene and changes the exposure to give it a better chance of standing out. We have a new bloom and drop shadow, a background blur effect that we can apply behind the UI to smudge out some of the details. And today, we just got a request to also clip out extra bright details behind something. So say somebody's got a torch, or there's some particular light just in front of you, and then you bring up mobiGlas, we'll clip that extreme bright highlight out. But there's a lag from us implementing the shaders and then getting it into the engine and getting it into the actual assets on the UI team. So we've done quite a lot of work there, and they're looking to try and integrate some of that into the UI pipeline, [which should] come in, I don't know, weeks, months. Definitely some of that we're actively talking about and improving.
And folks may want to check out, it was relatively recently, November or October, we did one of these Q&As with Simon Bursey, the director of UI, and the same question went to him, and you can see his answers. It's not the kind of thing that moves particularly fast or changes too much in three or four months, so the answers he gave there will probably still be applicable today.
TL;DRAli Brown most wants a solid 60 frames per second first, and then better global illumination, which he says is the engine's weakest area compared with other engines.
This probably won't be the answer everyone wants, but I just want frame rate. Frame rate's king for me. Until it's a solid 60, it's no good. So yeah, for me, it's frame rate. Once we have a solid 60, my interest will pique elsewhere. There are some areas of the engine visually that I certainly want to see improve vastly, but not before frame rate. That's the number one for me. If I had to pick a second, though, just to pique someone's interest: our global illumination lighting system is lacking. That's probably the area where we are furthest behind other engines. Everything else is on a par with or better than your average competitor. But our global illumination system is not where we want it to be. So that would help the visuals and really help the art teams as well. So that's what I'd like to see.
TL;DRThere is no magic bullet: Gen12 CPU optimizations come first, then GPU optimizations and graphics settings, with networking, server meshing, streaming and other teams' work also needed, plus clearer performance stats for backers.
Well, it's a different answer for every single person, obviously; different machines. Some people might already be getting north of 60 FPS all the time. Some people will never hit 60 FPS. So for me, it's three things. It's Gen12 first. Once we finish Gen12, which is our CPU optimizations, we shift our focus to GPU optimizations and the graphics settings, to give people control over how their GPU is being used. And then after that, the one part that I have no control over is just the rest of the engine and the game code, which we talked about earlier when we talked about XenoThreat. The particularly challenging areas of the game where we get a ton of players, or a city and whatever else on screen, or things like expo halls, sometimes can be challenging. This is where the rest of the company will have to take, or they are taking, their part of the problem on. But yeah, for our side, it's definitely just Gen12. And then immediately after that, we'll look at the GPU side of it to give you some more flexibility there. I really want to see 60 FPS after this. There will be situations, like I said, that will be more troubling, more difficult. But I'm hoping for a lot of the game, for most people, that's the target.
In addition to graphics stuff, there's networking stuff, there's server meshing, there's streaming, there's optimizations in the actual assets themselves, there's further LOD development, it's everything. If there's one answer you take away from this question, folks, there is not one thing that will get us where we're going. And that's the truth for any game on any platform, and especially true for one as big and as ambitious as ours. It takes the work of almost every team in every aspect of Star Citizen to get us where we're going, and that includes our performance goals. I just want to disabuse anybody of the notion that there will be a magic day where we throw out a patch and everything will be better. Just like with Vulkan, which we talked about at the beginning of the show, it will be implementation after implementation after implementation that gets us where we're going. And then, like getting used to cold or hot water, it will just suddenly be there, and it'll be where we need it to be.
One thing on that I'm going to add is something I spoke about at CitizenCon and I've talked about on Spectrum: I would like to improve some of our debug stats for the backers so they can understand their performance a little bit better, because at some points you're CPU bound, and a better CPU or more cores or faster cores would be what you need, and at some points it's GPU. And I want to more clearly display that to the users, or the ones that are interested in that, so they can better understand why their frame rate is what it is and whether there's anything they can do. And especially, obviously, if they're GPU bound, it does become fully under my control. Well, art as well, obviously; they're producing the content and we put it on the screen. But ultimately, the graphics and rendering is under our control. So when you're GPU bound, the graphics options we provide you and how our technology works will be all we need to get to 60. But if you're CPU bound, then it's a more complicated picture. But we're hoping to inform people, because there is a lot of misinformation and confusion sometimes about the performance of the game, because it has changed so much and it has been difficult in stages. I just want to clarify it for backers. No nonsense: this is where your frame rate's at. Hopefully that will help people understand and help them improve their frame rate as well, and see the improvements as they come, so they can see their GPU time reducing or the cost of the rendering side of the CPU decreasing as we release Gen12, even if they happen to be in XenoThreat at that moment and still not getting quite the frame rate they want.