Graphics and engine developers explain the Gen 12 renderer and Vulkan migration, which target CPU performance rather than GPU speed, and give a roadmap. They also cover upscaling (DLSS/FSR), HDR, night vision, ship light shadows, ultrawide UI, VRAM use, LOD pop-in and Linux support.
TL;DRGen 12 is the internal rendering architecture and Vulkan is the underlying graphics API replacing DirectX, with Vulkan scaling better across CPUs and threads, and further performance gains are expected in coming patches if all goes to plan.
Yeah, that's probably a question for me. Probably the best question to start with, since I'm working on it pretty much full time. For the people who still don't know what the difference is, we've already published a lot of videos about it and other people have talked about it. But Gen 12 is basically our internal engine structure, the rendering architecture for how rendering works. And Vulkan is the underlying graphics API, which one day replaces the DirectX graphics API. And the benefit of Vulkan is pretty much performance. It scales much better on multiple CPUs and threads. So, well, performance. We all love performance and we've all seen what it can do in 3.17. If everything goes to plan, performance will get even better in the coming patches. So I'm very excited for that, especially after we shipped one of our first milestones in 3.17 for Gen 12. And I'm working hard on it to get it done.
TL;DRThe first Gen 12 milestone in 3.17 moved the static brushes and objects on planets through the new renderer, with characters, ships and dynamic objects next and transparent objects still a large task.
So Gen 12, you can kind of split it up in two parts: all the scene rendering, like all the objects you see on screen, and then all the other rendering stuff like post-processing and compute passes and pretty much everything else. And the scene rendering was especially difficult to do, because we have hundreds of shaders and thousands of shader permutations and thousands of materials, and we need to make sure that everything works exactly as the legacy renderer. And we had a lot of issues and edge cases which you don't see during development. So just once you submit your code and it crashes somewhere, and then you take a look and see, oh shit. Okay, that's what happens. I forgot this. I forgot to add this. So I'm going to add this edge case. And so the first milestone was getting part of the scene rendering done. And for 3.17, that was the brushes. The brushes is what we call all the non-interactive static objects you see in game. And they're going through Gen 12 already. And the next step would be characters, ships, dynamic objects, and the planets didn't make it in 3.17 as well, though the objects on the planet, they made it in 3.17 as well with Gen 12, so happy about that. So once we have that, Gen 12 is almost complete. We still need to do all the transparent objects, which is still a huge task to tackle. But we started on that, and I'm very optimistic.
Gen 12 is strictly just performance, but of course, Ali also mentioned that in previous talks, it gives us more room for more fancy stuff, and we can use all the new fancy features in Vulkan, like ray tracing and what else there is. So not just performance.
TL;DRNew rendering technology can save performance and enable further optimizations, some features may cost performance, and they plan to make the engine scalable, with Gen 12 described as the first step in an ongoing series of optimizations.
Well, ironically, a lot of the new technology we have actually saves you performance. So initially, it might even go the other way. It enables a whole bunch of features with things like mesh shading and variable rate shading, and sometimes not even hardware technology, but internal. Because we've restructured the renderer, we can now do certain things much easier than we could before. So there's lots of opportunities now for optimizations that we may not have had before. So it might actually be a double win after we can implement some improvements. There's features that I know there's questions about later, about DLSS or FSR and these types of technologies, which were problematic to implement halfway through when we're doing the conversion to Gen 12. Then they become on the table again once we've settled on a single new renderer. I don't think it's the end of the performance improvements once we've got to Gen 12. But there are also a few that would go the other direction and lose us performance. Future battle: we'll see how fast we can make it first before we worry too much about how much we'll slow it down in the future. A big focus we want to do is to make the engine scalable, so hopefully any new shiny things we add can be scaled up and down, ideally, so that you can keep the game running on the lower spec but push it hard on the high spec.
And completing Gen 12 is just the first step to a series of more optimizations we've already planned out. So expect much more to come after Gen 12 is done.
Yeah, we've got it on the milestones as if Gen 12 and Vulkan are a concrete thing that is not done and then one day is done. That's not quite true. I mean, it's useful for scheduling to make it appear that way, but it's not as precise as that. It's a continuous set of features. Even Vulkan will introduce new performance, or it might detriment performance in some areas, might improve performance in some areas. And as we learn more, we'll be able to get more performance out of Vulkan. So it won't just be we're done and we never look at this thing again. But the milestones we mention are a point of confidence when we think, yeah, that's like 99%. The biggest chunk of work is done.
Yeah, we'll try to answer that. So the goal of Gen 12 in the first place is to improve CPU performance. So in case you run an older CPU with your old GPU, you might see an improvement, because we can better utilize all the cores of your CPU. But the more likely case is that you're bottlenecked by the GPU, and then Gen 12 won't really bring a benefit there. Vulkan might bring a benefit there, but you never know. We have to measure that. It might bring benefits in one case but be a drawback in another case. That's hard to tell. But especially when you're rendering in high resolution, when you're trying to do 4K, the older graphics cards will most likely be a bottleneck anyway, and then both won't help there.
TL;DRThe confusion is understandable because the main work is keeping powerful GPUs fed from the CPU, so the milestone mostly targets CPU performance, which has been the biggest bottleneck for most players.
Yeah, I mean, it's stupidly confusing. We're working on a renderer and a graphics API, but somehow they don't make the rendering or graphics any faster. It makes no sense. But the reality is a lot of the work is feeding these uber-powerful processors. We have these GPUs that need work, and it's hard to keep up, to give them enough work. And that's why it's a lot of effort for the CPU to keep them fed. That's why there's so much work to do on the Gen 12 and the Vulkan side of these graphics APIs and the renderer. But like we said earlier, Gen 12 will open doors to optimizations, and then optimizations can benefit the GPU, but the milestone we're mainly focusing on is improving the CPU performance, because that has by far been the biggest bottleneck for most players. And it's why somebody with a 3080 or 3090 might not be seeing the frames they want. So for most people, it is the CPU bottleneck. I mean, the range between the cards mentioned there, the 970 and 1080, is quite large. The 1080 is going to be significantly faster. Then obviously resolution, like Jonas said, is a big factor as well. If you're running a 970, then hopefully you're targeting 1080p and not trying to push too much beyond that. Otherwise, you will struggle. I can see there's many reasons why things would be confused in this area. I don't know what we can do about that. It is just confusing.
TL;DRGen 12 currently covers only the brushes and the render thread is over twice as fast, but the main thread is the bottleneck and the mixed legacy/Gen 12 mode adds some cost, so full benefits come only once Gen 12 is complete.
Well, that's easy to answer. In fact, I posted a Gen 12 update and progress like two weeks ago on Spectrum. And initially, I didn't even plan to do that, because I didn't expect any performance benefits of Gen 12 in 3.17. But due to the rising amount of threads and people talking about these magical improvements in 3.17, and people wondering where that actually came from, I felt like I had to give an update. So that's what I did. The thing is, Gen 12 right now is only the brushes. They're rendering in Gen 12. But the big caveat is that with our rendering right now, we're mostly limited by the main thread and not the render thread. So there are two threads, and it goes very deep into engineering already, I would say. So for everyone who can't follow me right now, I'm sorry. But basically there's a render thread, which, as Ali said previously, is this kind of thing feeding the rendering work into the GPU. That, with the old legacy renderer, is really slow, because every frame we constantly check every state and see what objects we can render. And Gen 12 uses a different approach by just pre-baking the state in a small object. And then when we render, we just have the entire state ready and can just say, GPU, here, render this object. So the render thread, I tested it myself, with Gen 12 it's more than twice as fast. But if you're not limited by the render thread, which at least on my machine was mostly the case, it doesn't give you anything. So that's why I didn't expect any benefits and didn't feel the need to give an update. But in the future, with full Gen 12, that's another thing, because right now we have this mixed mode. We have objects rendering with Gen 12 and other objects going through the legacy path. There is actually a cost to that on the main thread, which is the bottleneck. Which means, what I would have expected, Gen 12 would have made it actually a little bit slower. But that didn't even happen. So I'm happy about that. But once we can get rid of all the legacy stuff and have full Gen 12, then we'll see the full potential, and I'm very excited about that.
TL;DRA native Linux port is a high priority for one developer, so Hau believes it will come one day, and Vulkan should already improve performance through Proton and Wine by avoiding DirectX translation.
I can give a short update on that as well. So for the people who don't know, DirectX only works on Windows. And Vulkan is created by the Khronos Group, which also created OpenGL. OpenGL is more an old graphics API, and Vulkan is the new modern graphics API. And these kinds of graphics APIs also run on various other operating systems. Vulkan is known to run on Windows and on Linux and even on Apple with an additional wrapper. It's called MoltenVK. And Christopher Bolte, he's very keen on the Linux port, so it's up top on his priority list. So I believe one day we'll see a Linux port.
The other thing I'd add to that, and this isn't first-hand knowledge but second or third-hand knowledge, is that people can already run it using the various layers called Proton and Wine on Linux. They can run the game now. I think there was an issue with Easy Anti-Cheat blocking that. I don't know if that's been resolved yet, but that was on Epic, I believe, to resolve. But that does work. But going through these layers that emulate Windows has a major performance cost. Apparently it's playable, some people say. But the use of Vulkan, I've been told, allows it to skip the layer it has to apply to translate DirectX into something usable. So even if we just had Vulkan and didn't implement native support for Linux, we would still get much better performance on the rendering by using these layers, because it wouldn't have to emulate DirectX anymore. It would just have to emulate any other Windows calls, with the input and the sound and things like this. But obviously the graphics tends to be a huge processor hog, so not having to do a translation between Windows code and Linux code will save quite a lot there, we're hoping. So it makes that route more viable until we get an actual official, native Linux client running.
TL;DRDLSS or FSR support is a top rendering priority after Gen 12 is finished, most likely supporting both, but Brown promises no timeline and notes in-world UI will make implementation awkward.
Upscaling is a very common feature now. Every single console game pretty much uses it. A huge percentage of the AAA PC games use it. I don't want to say it's the future, but it's the current; this is the expected way if you want extreme resolutions unless you have the fastest graphics card. Even when you do, there's actually reasons that DLSS and FSR can exceed the quality of native resolution in many scenarios. There are compromises sometimes, there's artifacts in some situations, but on a stable frame you can actually get higher quality, because it actually supersamples the image beyond the number of pixels you actually have. So it's like a perfect anti-aliased image, or it can theoretically achieve a perfect anti-aliased image, which a native render cannot. So they're definitely the future, or in the next few years, the way to get to 4K and beyond. In terms of implementation, we need to just finish Gen 12 first. There's no point. By far the biggest performance improvement is the CPU gains we need to make from Gen 12. DLSS and FSR or an equivalent are top of the list. We have lots of other work related to the PU and Squadron that we need to get on with, but in terms of general render improvements, these are top of the list. So yeah, definitely. Like I said last time, though, I'm not going to promise one over the other, or timelines for them, or whether we even come up with our own one. Epic came up with their own one that is more similar to a hybrid of the two. Who knows? But the existing APIs for DLSS and FSR are theoretically going to be easier to implement than rolling your own one. So that's the most likely scenario, and most likely would be to support both. But we have to see when we get to it. And one thing that makes them a little bit awkward to implement in our engine is that traditionally you render your scene, you upscale it to 4K or whatever, and then you render the UI on top of that. However, our UI is very much in the world in a lot of cases. We have holograms in the world. We have MFDs on ships. We have floating holograms, holographic displays in the ships as well, mobiGlas, all these things. So they're not a flat 2D image projected on top of the render. They're actually part of the world. Which makes it problematic, because if you try and upscale very intricate text and UI, these algorithms tend to destroy it and do some evil things to what you really want to be crystal clear. So we already have a solution which splits our rendering so that certain parts of the UI, when they get to a certain size on screen or distance, swap to a different mode that renders them and makes sure that the post effects don't interfere with them. So we're hoping we can capitalize on that and use the same technique, so we can do FSR or DLSS on the main portion of the scene, and some objects get removed from that and composited on top. But that does make some things complicated for us. So that's a little bit like, ooh, we have to see how it's going to work. It probably won't be bug free in that sense. We'll have to tweak that till we get it right. But yeah, it's definitely something we need. Upscaling, for the vast majority of our users, I see as the common route. Some people want to have native, but I think once people have it, especially once they have the CPU gains, which you need, once you've got CPU gains and you can get 60, and you're like, oh, I can either have native 30 FPS or I can have upscaled 60 FPS, it might be an easier choice once they see it.
TL;DRHDR work is on hold behind other priorities, with remaining tasks around color correction, game post effects and monitor-range tuning, and the developer may resume it once back in the office.
Yeah, I talked to our developer who works on that. And right now, other priorities took over that task, so it's on hold right now. He's doing home office a lot of the time right now, and he has a super fancy HDR monitor in the office. Once he comes back, which is hopefully soon, and we're even going to move into the new office soon, which I'm super excited about, hopefully then he can take it on and finish it, because I'm very much interested in that as well. For everyone who doesn't know what HDR is, it means high dynamic range. Basically, it means colors from a monitor can be much brighter and more vivid than on a standard monitor like everyone's used to. And rendering, physically based rendering, which most engines do these days, already renders in HDR. And then as a post-process effect, all those colors get compressed into what we call low dynamic range, LDR. And that's what goes to your monitor, after some other fancy post-processing effects; gamma needs to be applied, but that's a story for something else.
So a bunch of support, I think, has been done. I think the last I looked, the tasks that were remaining to make it fully finished were three of them. So we have color processing, which is something the artists apply where they want to set the mood of a scene, like a blue-tinted sky. And they don't tint the actual sky when they want to achieve a mood. They render the sky physically accurate, and then, if you imagine, a bad example, every nighttime scene in a film is filmed in daylight and then they just make it blue. That's an exaggeration, but that's the approach. So sometimes the artists use these things, usually quite subtly, but that process is currently mapping one low dynamic range color into another low dynamic range color. The process becomes a bit more complicated when you're dealing with high dynamic range colors. When they say they want to turn a white sky to blue or a blue sky to white, it gets more complicated. There's now millions of shades of white, millions of shades of blue. Did you just mean the very bright blues or the ultra bright blues or the extremely bright blues? So that process needs to be changed, and it involves changing the art pipeline a little bit and how they author the color correction. And the two other issues: we have what we call the game post effects, which are not the critical rendering ones like color correction or our lighting and blurs and things like this, but like when you've been shot, blood appears on the screen, or you're blacking out. These types of effects all run in low dynamic range, so they all need to convert to high dynamic range so they don't lose quality. Otherwise, they'll appear very dull if we don't do that. And then the final one is the more open-ended one. When people say this game has a good HDR implementation and this game has a bad one, like Silvan said, we already render the game in HDR. The problem is even the HDR monitor, it's high, but it's not infinite dynamic range. If we had infinite dynamic range, we'd just give you the image we rendered, but that's not the case. What you have to do is query the monitor, find out what range it's got, and make a best guess of how bright you think you should render the image and how you should color correct the image, and then probably also give some sliders for the user so they can tweak it themselves. That can be a hugely subjective point of view, and like I say, different games get either praised or beaten for how amazing or terrible their HDR implementation is. But it's really just that last part in the pipeline that people get upset over. So we need to just balance that and make sure we expose the right knobs and tweaks so everyone can get the image they want on their monitor.
TL;DRNo night vision is promised; Brown jokingly says it needs a design decision from Todd Papy first, then explains technical approaches such as approximating infrared by mapping artist colors with overrides for special materials.
Last time this came up, you promised you were going to lynch Todd Papy. So I think you should follow through on that promise and get him. Because we need technical ideas and thoughts on that, but we need the design. Is night vision in the future the same as now? Is it infrared? Is it based on some other form of technology? What I've mentioned in the past is that if you want to do it really well, then you emulate what we call spectral rendering, which is an incredibly complicated field. But there's elements of it you can simulate, where you can identify the materials that have a nonstandard way of reflecting infrared light, which would be anything that glows in the dark, or appears to glow in the dark, on an infrared camera, like reflective gear... I can't think of any good examples right now. But there's certain materials that don't reflect as you'd expect, so they need picking out especially. Otherwise, it doesn't look right in night vision. Otherwise, it's a type of filter. You need to take your colors and... It's a complicated topic. It's hard to break down. But what we see as red, green, and blue is obviously not the true picture of what happens in reality. In reality, there's millions of wavelengths of light, infinite wavelengths along a spectrum. And infrared is just looking at one portion of that spectrum. But the artist doesn't model what happens in that different spectrum. They just model what happens in these three specific spectrums the human eye can see. So we have to map or guess how we get from the artist's colors to the infrared spectrum, or whatever spectrum our imaginary night vision might be. And there's ways of approximating that. Or you can author custom textures if the artist can tell this is exactly what it looks like in infrared. But we clearly can't go and remake every texture in the entire game to show how the infrared would work; we'd double the textures in the game. So you normally do some type of automated mapping that takes a best-guess approach, and then you override the particularly special case ones. But yeah, it's all going to start with Todd and the request.
TL;DRLarge fleet battles are mainly CPU bound, which Gen 12 should help by an uncertain amount, and they plan to render ship interiors cheaply using environment probes to cut lighting cost.
J
JonasEngine Programmer
Right now, you will very likely be CPU bound in this situation, because so many ships all need to go through the render thread alone. But with the upcoming Gen 12 changes, it's hopeful that the changes we make will very likely improve that, at least from the rendering side, because we can distribute the work more evenly across your CPU cores. And that should help there. But of course, there are other systems besides rendering that you need to keep in mind, for example AI or game code or physics or whatever, that also take their part on the CPU. So Gen 12 will, of course, improve it, but it's hard to say how much.
I was going to say on the lighting side, because you were mentioning that in the question as well: the lighting tends not to be a big issue in space. Obviously, space is mostly pretty dark. Not too much light going around there. We obviously can potentially have lights on the lasers, but they're also quite easy to LOD out if we were particularly struggling for performance. Ironically, in a big ship battle, one of the things that hurts us quite hard is the interior of the spaceships. We don't have a great way of not rendering the interiors of spaceships. Obviously, if you can't see a window, then we cull them out from very far away. But if you can see into the spaceship, where you have something like the 600i with all its glass, you can't just not render it. You need an approximation of what's inside that room, and that's quite difficult to make. You can't just fade it to black, because if, say, the 600i has a bright interior and it just fades to black, it looks really bizarre. So one thing we were planning: we have these cube maps, we call them environment probes, which are used in the lighting system to capture the environment and then reflect it as a substitute for more advanced ray-traced reflections. So we were planning on using them to render a low-quality version of the room. It would be indistinguishable at the distances we're talking about, but would be the correct colors and have the correct geometry in there. So that would give us a way of basically culling all the interiors of the ships when we've got a massive fleet battle going on and we don't really care exactly how the interior looks, just as long as it's the right color and rough shape of the geometry inside. So that would help, and that would save us the lighting cost of all these interiors. But I think largely it's a CPU issue, like Jonas said. So Gen 12 would definitely address part of the problem. We'd have to profile what the physics and other friends are doing. I'm sure they'd have optimizations to do as well.
J
JonasEngine Programmer
For everyone who's interested in how these interiors could be rendered, just check out some footage from the latest Spider-Man games on PlayStation, how they do it. That's an example of how you can do it.
TL;DRThe interior-window shader already exists in the engine as a standalone shader and the technology works, so artists are now implementing and testing it, and the speaker is optimistic it will be seen very soon.
I think that's exactly what Jonas mentioned, I believe. So we already have this kind of, what we call it, interior shader in our engine, just as a standalone shader right now. And I actually didn't know about it until a month ago, when I saw it in-game because I had to fix a bug on it. So I've seen it live in the engine, and I was already impressed. It looks pretty damn cool. And we're already experimenting with it in engine right now. So I'm optimistic that we're going to see it very soon. But it's mostly the art side now who has to implement it and test and iterate on it and make sure that it looks good. From the technology side, everything's working, looking good.
So that's at the point where it's ready for artists to start playing with it, and then they're going to discover all the bugs and all the problems with it, and then they'll come back to you to fix it.
Exactly, yeah. I mean, they're already playing with it, and I'm excited to see it in engine. It's definitely going to give some more life to those buildings when you can actually see inside.
TL;DRShadow-casting ship and helmet lights are hard because the cost is difficult to control, so they are looking at ideas such as a user-adjustable limit on dynamic lights, with nothing committed.
We've gone back and forth a few times internally. It's controlled by the art setup, so I never know for sure what has and hasn't been set up on each spaceship. But the problem with the feature is it's quite difficult to control the cost of this. So one spaceship floating around in space, if you put shadows on its lights, no problem at all. You park 50 of them all together pointing at each other in a really busy location, and you all get out of your spaceships and jump around in front of them, stack up boxes and all the crazy things people do in the PU, it's going to start costing a serious amount of GPU to do that. So we have to figure out how we can add it yet keep it under control in the more difficult scenarios. It's been suggested that we can just fade these shadows out when they're too costly, but that actually becomes much more complicated than you think, because what you don't want to happen is that as you move your camera around and look around the scene, things are popping on and off all the time. You want a stable, consistent visual. But we've got some ideas of how we might handle that to try and avoid the issue. And we might apply the same idea to all types of dynamic lights, from ships, your helmet light, your flashlights, and muzzle flashes on guns, which is another request, to have shadows. All these things; the muzzle flashes especially are something where it'd be great to have shadows. But if 50 people are firing in a room, then maybe you can say, oh, we don't care so much. But it's things we could also mostly provide to the user, to say, do you want these? How many dynamic lights do you want to support? Let them choose if they want to pay that cost. But there are also some internal budgets. We have basically a texture pool where all the shadows go into. Once we've exceeded that, there is just no more space without sacrificing the quality in the rest of the level. So I'm hoping we can come up with some type of system where, how many dynamic lights do you want? Maybe eight might be a default, or four or something like that. And then you can scale up to whatever you want if you want to take that hit when you have a really busy scenario. So hopefully. But the ship ones are particularly problematic because they can cast thousands of meters and cover huge areas, which means if you cover a whole space station, there might be a few thousand draw calls in that space station, which are instructions for the GPU. You've just doubled that by pointing your light at it. And 50 people pointing 50 lights at it, 50 times the cost for that space station. That's a big deal.
TL;DRAli Brown wants exterior ship cameras, but not photorealistic ones because of the frame-rate cost, and favours a cheap depth-camera or LiDAR-style view if design doesn't ask for photoreal.
I'm also going to point you to Todd Papy or someone again, or a design director, maybe John Crewe. I've actually been pushing for this. I want this. But with one giant gotcha: I don't want a photorealistic camera, mainly because of cost. I think you'll pay a frame rate cost when that happens. If you play a racing game like Forza or something like this, they'll have their wing mirrors and their rear-view camera, but they have heavily optimized their game to make that work, and they have heavily optimized what happens in that rear-view camera. You'll notice they never look as good. We have a hugely complicated game. There's much more than just a road race arena: orbit, gas clouds, we have all sorts going on. So to render all of that again periodically in an external camera, at worst case, it could halve your frame rate, which we don't want. So more sensible for me is that we do some type of LiDAR, depth camera, the type of thing you actually get in cars in real life. And I think that'd be far more useful, to be honest, because imagine you're landing a giant spaceship on a space station: you don't really have any sense of scale. It's an enormous beast, and you're landing on what might be just concrete or tiled metal panels. You don't know how close you are. So I think something that really gives you a great sense of depth, even if there's not a lot of interest there. Or a better example might be landing on a snowy mountain. You won't be able to tell anything if we just show you a white picture underneath a spaceship. So I think a depth camera or a LiDAR or some type of holographic, some type of nice imaginative non-photoreal render, would be very cheap and be cool. So yeah, I'd love that. As long as design don't ask for photoreal.
TL;DRUltra-wide HUD and chat scaling is mostly the UI team's responsibility, not graphics; a meeting with the UI team is scheduled, and graphics has some ultra-wide post-effect and performance work of its own.
So I'm using an ultra-wide monitor myself, so I know about the issues with Star Citizen. Unfortunately, we as Graphics and Engine don't have too much to do with that. It's more a question for UI and how they implement it, especially in the Building Blocks system, how the UI scales with all those different monitor resolutions. I'm not sure. Maybe Ali knows a little bit more about that, if there are priorities on that?
There's a few graphical things with ultrawide that we sometimes need to fix or improve the performance of, or sometimes the quality; maybe our post effects aren't always amazing in super wide. And also, if you're super wide, depending on how we deal with the aspect ratio, you can have more objects to render and obviously more pixels to render. Gen 12 and other optimizations like DLSS and things we've talked about would all help there. But in terms of just making it a fun experience, with where you position the HUD and the chat windows and stuff, it's all on the UI team in the end, and we don't really have any control of that.
TL;DRThe game caps texture memory because more VRAM rarely improves visuals, but after Gen 12 they want to expose memory settings to players, and a high-res texture pack is a possible future option that can't be promised.
Yes. So the first thing I'd say is you don't want to use 24 gigs just because you can. You want to use what you need. But the question is, what quality improvements could we make, which way could we use those 24 gigs? At the moment, the system basically assumes it wants to achieve the resolution you ask for. So if you ask for 4K, it's going to spend a lot of memory achieving 4K. It looks at what memory it's got left on your graphics card, and it says, OK, I'll use all that's left on the meshes and textures. And it will scale up the texture budget to about three or four gigs; I think that's the internal limit of how much it'll use on textures. We could uncap that three-gig limit and put it to 100 gigs or whatever, and it'll just try and load all the textures in the game for you. The only reason we haven't really uncapped that is it doesn't usually lead to any visual improvement. Three gigabytes tends to get you every single texture we have on screen loaded at the highest quality we have available. So raising it just risks a little bit. We might get some streaming issues where your hard drive just starts churning away. And maybe you've got a fast hard drive, it's OK. But maybe you don't have such a fast hard drive, and you're wondering why the game runs slow, because it's trying to load the 24 gigs of textures you've asked for, which you don't really need. That's the situation today. Where we want to be, though, once we've finished Gen 12, is to expose some of these numbers to the player instead. So you pick your resolution, and it shows you how much memory is left, and you pick how much texture resolution you want to use. And then you can down-res, up-res. And then if you want to hang yourself and put 24 gigs of textures, maybe it does look better, maybe it doesn't, but we'll give that to the user to pick how they distribute the memory. One thing that would be nice, which we can't promise, but it is definitely a future option, is to have a high-res texture pack. We author all of our textures at double the resolution we actually ship them at, but there's several issues in trying to achieve that at the moment, mainly that the size of the build would obviously go enormously high, and that has costs associated with it as well on the back-end side; there's some money. So we also have some tasks in the pipeline to reduce the size of the build. This is stuff that's a higher-level decision than we can make in our team. But maybe if we have the textures available and we have the technology to use them, and we also save a bunch of memory on the build, maybe we get to a point where it's more viable to have a high-res texture pack. And then the people that do have 20 gigs or whatever of memory will have a very real reason to use that memory. If you, say, doubled the resolution of every texture, that would be four times the texture memory. So our 3-gig texture limit might go to 12 gigs instead, at least. So 12 gigs, once you add on 4K and a few other things, you easily get to 16 to 20 gigs of memory. So this is something in the future, definitely. There are some technologies as well in the future that we might use that could eat up some of that memory, things like ray tracing and SDFs, distance fields that are used for various pieces of technology. These all consume extra memory. We just have to make sure the game can still scale so that the people that have six gigs or whatever can still run the game sensibly.
Well, interestingly, a lot of old games used to let you max out beyond what your GPU could do. I think some games now have taken the approach that we're talking about here, where if you max them all out, it will not let you. If you want to run your game at 8K and have all the textures loaded in the world, it's not going to happen. There's no space physically there. So it will force you to live within the constraints of your GPU.
We have at least taken a look at that. But we already have custom technology in place to improve the memory transfer to the GPU. So it's unclear if we can gain a lot of performance from that. That's something we will have to take a deep look into, measure, and check where it makes sense, if it makes sense. And then it will probably be implemented. The diplomatic answer here.
TL;DRDraw distance for foliage is set by art and should be scaled up as Gen 12 and Vulkan improve performance, and discussions about grouping objects for dense distant forests are ongoing, but no concrete plans exist.
Yeah, I'm seeing that too. The thing is, that's mostly decided by art, how they want to set it up. Of course, it's measured based on performance. But Gen 12 and Vulkan will improve performance, and I think we will scale it up at one point. But I'm not sure what exactly is planned there, if there's anything official yet.
Yeah, plans, not concrete, but definitely discussions with the Planet Tech team about how we can take things one level higher. At the moment, basically, each object is considered an individual object in the scene. It's spawned and we manage the position of it. We do have some small-scale grouping. We might group five or 10 objects together. But what you need if you want to, say, render an entire rainforest is the ability to have millions of trees. And just having the concept of an individual tree exist means that becomes beyond the reach of a sensible-speed computer. You need to have the concept of: this is a patch of forest that might be a kilometer wide or 10 kilometers wide. And the trees either never existed individually, only as a group, or they are being spawned extremely quickly and then merged into one Uber clump that we just have to submit to the GPU, or not even submit to the GPU, but just render on the GPU without any CPU involvement. But the current approach of having one object constructed and then sent to the GPU each frame just doesn't scale quite to the level we need. Gen 12 will give us a big boost on what would be possible with the current system. But there's plans for the extreme modes, we'd call them, like the rainforest, where you can just see for miles and miles of trees. That needs something slightly different for that to work.
And all these objects on the planets are already using a more specialized path to be rendered much quicker than our usual objects, like all those brushes. But as Ali pointed out, if we want to have really dense, lush forests like you've seen in Microsoft Flight Simulator, there's definitely a lot of room to improve, and it's definitely something we're looking at and aware of, and hopefully we'll implement something in the future for that.
TL;DRNext up is finishing Gen 12, mainly transparent rendering, then Vulkan, with other graphics work on Squadron 42 features, jump points and water effects, and possibly DLSS/FSR-type approaches later.
Well, as I already pointed out at the beginning, we want to finish Gen 12. So there are a few more milestones we have to finish. The transparent rendering, that's the biggest one. And once that's done, I expect all the other things which are missing, which are more like a few compute passes, like Planet Atmosphere stuff and compute skinning and a few other things we've probably overlooked so far, to be finished. Then Gen 12 is done, and I'm really excited to see how much performance we can get from that. After that comes Vulkan, which is not lagging too far behind, because in general we're developing Gen 12 and Vulkan in parallel, and Vulkan is just lagging a little bit behind. So once Gen 12 is done, it shouldn't take us too long to get a first version of Vulkan working.
We've got these guys on Gen 12 on the engine team. We've got about half the graphics team on Gen 12 as well. The ones that aren't on that are working on a number of features. Mainly, there's a bit of a backlog for SQ42, the critical features we need for that. They'll obviously feed into the PU as well. There's very little that would be specific to Squadron; it's just who is the prime need for each feature. But there's a few features in the works for them. Some research for longer-term features like jump points, the graphical implementation of that. Things like improving the water effects are on the near-term goals as well, and unifying them. We have a planet ocean shader. We have a water volume shader. We have a river shader. These things all need work and love, and to be integrated with the VFX as well. In the transition of our engine over to all the improvements we've made with planets and stuff, we've lost splashes and dynamic waves and stuff like this. We need to reintroduce these types of technologies. But mainly we have half the team on Gen 12, and when that's done, they'll be on Vulkan until that is running buttery smooth. And even then, that team might then look at the DLSS/FSR type approaches, and only then will they join the rest of the team on the general graphics features.