David "Bone" Gill

That's definitely the plan, yeah. But there's a lot of back-end work that we need to look into, and especially with server meshing coming in. So a thing that happened for the 3.23 work that we've been doing is that we started talking about, okay, we want to get these pins in as part of the star map now, and you should be able to place them and share them with people and all of that jazz. Then we looked into it and, well, it was quite hard to do and we're getting server meshing. So everything's going to change and it just wouldn't work with the future.

The technical challenge is immense. The pin itself, drawing a pin on the map, fine. Persisting it in this sort of super complicated structure that we have is tricky, tricky to say the least.

That's not to say we won't work on it.

No, we're working on it, yes. It's just not that straightforward.

Yes. Emily might have some answers. I was just stealing her thunder a little bit.

Yeah, so there are a lot of plans and a lot of thoughts that we have. We obviously want to use the AR space for markers. If you're going into a hangar, have a gate bracket so that you know the bounds of where you're trying to go. A ladder sort of system so you have a guide to where you're trying to go. And then also we want to make use of the radar display that you have as well, to actually utilise the area map tech again to display your ship and the ground underneath you. And then obviously if you have a landing pad, it will show you your landing pad, and if you don't have a landing pad and you're just trying to land anywhere, it'll have a line with a shadow kind of thing and it will sort of guide you to whether you can land there, whether it's flat enough, that kind of thing. So these are the sorts of things that we're thinking about doing.

And that'll tie into how we want to show terrain and things like that in our map system, because I want that to be a thing.

That's the thing we always find out in the last moment.

The geometry for planets and stuff is streamable, but when you start talking about being able to see other players, other locations and stuff, and whether they're exactly on your server and all this kind of stuff, that's the stuff that gets complicated.

But actually the interesting thing to mention is when we do account for, in our map system, things not always being streamed in, maybe the spaceport on the other side of the city isn't streamed in so you can't really see it or whatnot. When we make that work across the board, the map should work regardless of whether things are being streamed in or not. It should be totally independent of the entities that are existing. It opens us up to look at maps of far away places anywhere that we've cataloged them. So if you're on another planet, you might be able to see the map of what you've explored and not even be there. So that's an interesting byproduct of that, two benefits in one.

I think we have a line to tread.

Yeah. So we want it to look cool, right? But we want to be able to use it at the same time. So there's a balance to be had. We're not going to release something that looks like a vanilla web page, that's just black text on white background for all the UI. For sure. See, there's the extreme where you've got all this chromatic aberration and bloom and fog and effects and stuff going on, and it's not particularly readable and it's difficult on the eye. So we've got to find a balance where it looks cool and it's diegetic and it's in the world and it looks futuristic, but ultimately very usable. You've got to be able to use it, and it's got to not be bad on the eyes.

No, and I think with the mobiGlas in particular, because I think that was maybe the worst offender in this case. It's supposed to be this holographic thing, so it got a bit over the top and we got overexcited. It can look even cooler. But no, we very much appreciate the feedback, and I think everyone's very happy with the results that we've got now and it's coming out really well.

But we're still dialing that in, to get the tone right.

It'll be a continuous process. And there's also some graphical fixes that we've fixed recently as well. I think on some of the low-res render-to-texture stuff, there's actually some color correction weirdness going on that graphics seem to have fixed. So that's probably going to increase a little bit the brightness of some of the elements, because now the colors are consistent between low-res and full-res render-to-texture. And there's also some sorting issues that we need to work out in the system with regard to 3D planes and 3D objects. There's sorting issues that we've sort of fixed, but there's a proper solution we're going to do post-3.23.

So it sounds like some of it is technical, some of it is bugs, things not working the way that they're intended, and then some of it is pushing to find this happy medium.

Yeah, finding a balance of good art but good usability.

Because if there's a 3D plane in the world that's behind another, the thing that's trying to render a drop shadow or whatever, it's the thing that's over it. And when it's sorted incorrectly, you're going to null out any of the benefits of the backdrop or the drop shadow or whatever we put in.

And some of the diegetic UIs are in the world, so they receive the same world post-process effects, like chromatic aberration and the bloom and all these kind of things. So it's trying to combat them sometimes to make things perfectly readable, but because we're doing a diegetic game, because these things are in the world, you can't just sort of say, hey, don't apply these effects, because it would look really, really strange. It would look really out of place, like a game just putting stuff over the top of the screen after you've rendered the game world. We're not really after that kind of look. It's meant to look like it's part of the scene. So these kiosks or even the mobiGlas are part of the game world.

No, for me, I guess it's quite exhilarating in a way, but I always wish there was just one more week. Just let me play around a little bit more with the materials. Let me put some animation in and make it a little more polished, or more feedback. Like right now, I need to put in the routing feedback. So when you press R, right now it silently fails and you're like, why isn't it doing anything? So if you have that, it's just a small bit that makes it feel a bit more polished.

And then when you fix that, there's another thing you think, oh, it'd be cool if we put this in as well. It never ends.

There's another thing. You can always keep improving, and that's just the process of design, right? So if we kept adding this one more week all the time, we would just be driven insane, and nothing would ever release.

Well, readability, yeah. It's not as straightforward to solve as we thought it would be. We have a constant battle with this, partly to do with exposure compensation, i.e. when something's super bright, your eye adapts to that brightness and changes the brightness levels of the scene. Obviously there's the problems of drawing some coloured text onto a black background, i.e. a starry night, and then all of a sudden you're on a snowy backdrop. How do you compensate for that? We've still got a few more irons in the fire, especially on vehicle HUDs, about what we're going to do to solve this, and not just drop shadow. I think what we've got on the lens now on the AR stuff is really going in the right direction. It still needs tweaking, but we're pretty happy with that. But we've still got issues with the head-up displays that are on vehicles, and the exposure compensation on MFDs we still need to look into. But yeah, it's weirdly tricky. The more diegetic you are, the more you sort of buy into the idea that these things are in-world and you're not cheating and you're not just drawing over the top of the screen with some flat UI. I used to work on driving games years and years ago, and you draw the world and you draw the car and then you draw a HUD flat over the top, and that would be crystal clear and you could see your lap time and stuff, whereas this is entirely different. As soon as you put stuff in the world, it can be at angles and the sun can shine on it or a big shadow goes across it. Even your mobile phone has a lot of exposure compensation. Your eyes have exposure compensation, but phones have exposure compensation, so if you bring it out in direct sunlight, it's super bright, and if you get it out under the covers in bed, it's all dark and stuff. I don't know why you'd be getting out your phone under covers in bed, but maybe that's just me.

But in the future, yeah, we do want settings for colorblindness and we do want settings for being able to set font sizes and so on, but that's...

Accessibility is a completely offshoot of this where, yeah, I think it's one in 12 males have some form of colorblindness, and some of the colors we've chosen don't really work with people that suffer certain types of colorblindness problems.

No, and again, we have all of the different manufacturers and so on, different styles for every ship. So it's like, oh yeah, this ship works great. And then another one, you didn't at all understand that you were dying or whatever.

Well, I blame the manufacturer. But in all seriousness, yes, we're going to put the settings in that will combat that problem, but also accessibility options for changing the font sizes. This obviously has an impact on the sort of graphic design look that we initially have, but essentially it will become usable for people that have issues reading the font sizes or the particular colours that maybe that manufacturer uses. So yes, yes, yes, and yes again for options to help people with colorblindness, or for font size accessibility options as well.

In addition to that, as we start really converting all of the legacy UI into Building Blocks or new tech, it's all fully data-driven now, so that makes it much easier to just transform values where we need to, like inject a scaler if you want to have bigger fonts or lower-sized fonts, and also colors. It's all data-driven, so it makes that much easier. It's not static anymore.

What's good about it is we're not talking about Building Blocks 2. We are happy where we're at with this, right?

Building Blocks itself has grown. Building Blocks continues to evolve. You continue to add new functionality.

We need to optimize it greatly. There's still a lot of optimization work that we need to do.

That's kind of down to, weirdly, it's like a victim of its own success. This happens a lot with our game, where we go, oh, we've optimized entities. Someone puts 1,000 entities in and it's running slow. Okay, well, we've made it run twice as quick or whatever. Oh, that's great. We've now put 3,000 entities in. And then within a couple of years, it's now 300,000 entities in. It's a battle. Well, a battle's a bit negative, but it's a virtuous circle. We speed things up, you can do more stuff. We speed things up, you can do more stuff. And it's the same with the UI. So we speed Building Blocks up. We started off with tens of nodes, and now we talk about hundreds of nodes, and we're looking to do thousands of nodes in like a millisecond. So yeah, there's lots of optimizations, but doing the optimizations just means we can do more and more cool stuff, like lots of little widgets and stuff, all moving and animating and stuff like that.

Yes. I can't remember the effect, but there's a desaturation effect, right, where everything sort of goes black and white and everything goes a bit hazy. The UI goes black and white with it, because it's in-world, so at that point it loses its colour. Or there's things where you get this extreme post-process effect. I think it's like when you're too thirsty, whatever the word is for too thirsty. Dehydrated, thank you. Yes, we get this really chromatic aberration and it affects the UI because it's in the world, like you're trying to read your phone and you really need a drink.

And as you've seen for the shopping, now we moved the UI to be in-world as well, and that just made for such a big challenge to implement. It is a tough thing to do. I think that's the bottom line. But we are adamant about this: this is our way. This is how we do the things. And it should feel real. It should feel tangible.

I think I might have caused it. I might have dropped myself in. Yeah, we talked about how we show all the points of interest. It turns out there's a lot of interesting points in the universe. It was about filtering, right? So filtering what you want was really tricky. So it just didn't happen for a long time, because you can't filter by distance, because you obviously want to go to places far away. So yeah, it was an extremely tricky problem to solve, and it's taken this long to actually get to a usable solution. When we talk about the markers, we've had long discussions about, well, let's merge markers together when they start overlapping. So do you have enough information? One of your friends with a planet, do you get the information about that kind of stuff? You don't want the information where you just get a blob of overlapping text, that's no good, right? Do you make them avoid each other, or do you make them like, hey, your friend's here and also there's a planet here? How do you describe that as a bit of UI? Deeply tricky problems to solve in a sort of systemic way. Everything needs to be systemic. We can't just say, oh, right, here's just the ruleset, because the ruleset is endlessly complicated, because we have an endlessly complicated game.

The systemic solutions are what take so long. It would be easy enough to go in there and stick a proverbial patch on something and just stop it temporarily. But everybody knows the most permanent thing in any video game is a temporary solution, and it would end up just blocking and breaking a bunch of things down the line. We always look for these broader systemic solutions that can keep solving these problems as we add new things.

And when we don't, we always get tripped up. If we sort of cheat, say, oh, well, let's just come up with some bodgy rule, right? We always get tripped up by it, because something else comes along the way. And again, when I've worked on more traditional video games, let's say driving games, if you're going to draw the world, you only have to draw enough of the world away from the track. Whereas in Star Citizen, you can go anywhere, and all these things interact with each other in a systemic way. You have to find a solution that works correctly all the time. We've really made a challenge for ourselves to do that. Love it, though.

I think if you ask the same question about other things, why it's been there and taking so long, and it's been there for years, it's because it's an old system. We don't want to touch it with a 10-foot pole. It's a waste of time. That's because we have this new system that we're working on, and all the focus ideally should be directed towards that. We're in the middle of development.

And for this question specifically, the marker system has now been reworked. We have a new marker system, so it's no longer in Flash. We can actually start looking into this. We now know exactly how things are done, and we have our best men at it.

So this is someone being a bit funny, I think. The external starship view has no UI at all. That's it. But I don't think there's a plan to do that. We won't go that far. But we are working on the HUD experience for ships, for sure. That's a big one. And it's going to be cleaner. And we're going to add more options for people to turn off things that they don't want to bother them when they're flying around. Also, just looking at the new flight mode that we're getting in 3.23, it's a very clean experience. It's very nice to just fly around in flight mode and enjoy that. So that's not quantum, because then you get the markers still, less markers than before, but then you can go into flight and just do the thing.

So I was thinking the other way around, which is, can you see the UI from outside the ships? As in, you can see through the cockpit, and if someone's flying a ship and you're EVAing behind them, you can see their UI. Because that's like being on the bridge of a ship, right? You can walk around and see what people are doing on their various consoles and stuff. So you could EVA, take a sneak peek, check out what you're doing.

What you're doing there on your mobiGlas.

I suppose the most straightforward answer is the slider, which goes from the standard 16:9 aspect ratio that you'd get on one regular monitor, and then you can slide it to whatever aspect ratio you're running at. So if you've got three monitors, you're going to be 48:9 effectively, right? So if you've got the corner pieces, like the interior map and the thing in the top right and the health bar, you could effectively put them right to the corners of monitor one and monitor three and be like this all the time. But it's on a slider, so you can choose somewhere between 16:9 and have it on your center monitor if you've got a 16:9 monitor, and then move it all the way out to however many monitors you have. Or if you've got a 21:9, it's probably just going to be a small change, but it's user-defined, that thing.

Yeah. And that's in already.

I'll just sort of give you my opinion, which is this is sort of our first drop of, and obviously we're going to do the vehicles as well, but we've got the first drop of the new visor and lens layouts. And what we want to do is make that as good as possible for everyone, as much as possible, right? So in an ideal world, you'd want nobody to want to customize it because it's perfect, but we don't live in an ideal world, so we're open to design decisions from there, I suppose. But ideally we get to a position where the design's good for the absolute majority of cases, and it's not people going, I just hate this thing, I want to turn off the interior map, or I want to turn off my health bar or whatever. If the health bar is not as intrusive, or, I'm just paraphrasing, the other thing that just comes on and just notifies you something and goes off, then maybe it's not so intrusive that you're going, ah, I must turn this thing off. If we get as much right, then you sort of reduce the need for that much customisation. But presumably the customization question is still open-ended.

Yeah, I guess to a degree, right? Because I fully agree, we are working on this UI actively. This is always improving, and we are making it as good as we can. But bottom line is it's always nice with customizability, and I think that would be again for the far-flung future, really. We need to make sure that we have all the UI at its best. And then we can start looking into, okay, now it's worth giving people the power, because we've done our best now, and let's allow them, and also let's allow ourselves to take the time to look into something like this, because it's also...

Which is equal on this sort of accessibility options as well. If you get it right, then hopefully a lot of the accessibility issues are not as strong. We will still have all these accessibility options for changing the text height and stuff, but it takes you away from the best version. You might make it more usable for certain individuals.

There are certain aspects like turning the minimap on or off. You want to allow for that, because maybe you don't want to see the map all the time. You want to use the signs. You want to let yourself discover the environment without tools. It's kind of like letting go of technology in a way, going off in a forest or something. But having that there, at least initially, that allows us to gather a lot of the testing. Is it going to break on a specific machine or a setup? And so we can gather more data initially, and then we could figure out maybe a hardcore mode for the HUD, where you don't have certain things. It's just wipe everything away.

Yeah, turning things off is certainly something that's going to be easier.

I feel like I would want to play under that mode sometimes.

It's a lot easier to turn the minimap off than it was to write it in the first place.

So it's really about tooling at the moment. We do have tools for making the UI, but they're not very user friendly. The two guys previously would definitely attest to this. So we want something that's definitely more designer friendly, definitely more artistically friendly as well. And it's a complicated business, because Chris is really into very 3D UIs, and we've all seen the stuff that Zane produces, which is awesome, but he's extremely technical. And what we want to do is lower the bar so we don't need extremely technical people. Anyone can pick it up and make cool-looking stuff. You still need artistic ability and to understand UX, but at the same time we want to make this really achievable for the broad range of designers and artists.

When we say we want to make it more broadly available, that is because we are, as a company, as a team, fast approaching, if not over, a thousand people, with lots of developers, all with varying levels of technical abilities. If everybody could program UI, everybody would. So making these things accessible is super important. It's not about, well, let's not hire people who aren't technically proficient. You hire people who have their proficiency here and here and here, but you want to build a thing that doesn't require them to also have a UI programming proficiency, so that you can spread this out. That's how we hit this at scale.

But also just do stuff quicker. If you can try stuff out and go, hmm, I'm not quite so sure, try some other things out, and you're not struggling with the tools, you're not struggling to author stuff, and you just go, here are some things. Right, yeah, that really suits my feature. You can move on really quickly. You can produce more work if you've got better tools. It's like trying to build a house with a tiny trowel or something. No, you want some heavy machinery that can build this stuff together. So the better the tools, the better people can work, and it frees them up to do more creative stuff rather than just screaming at their computers going, these damn Building Blocks.

I'll start. There's plenty of things that Bone could say on this. But I think the important thing from my side is that we've rolled it out from something that the core UI team used to use, and we were working out how to use it as well. It's got to a point now where it's good enough for the rest of the team to use. So we rolled it out to game designers across the company, some of the artists, and that kind of thing. So it kind of makes our jobs easier, I suppose, because it's not just six people trying to get the UI done for the whole company. Now, those particular teams can take control and they can make their own kind of demos and functional versions of stuff. So that's a really big thing that it's brought to us. I'm sure Bone could go a lot into the technical things that have changed over the past year as well.

The past year, I think the major thing that we've been working on at the core tech level would be adding 3D. It's a big feature, and as soon as you jump from drawing some text and some images to, hey, I want to be able to draw models, I want to be able to draw characters, vehicles, and all these kind of things as part of the UI scene, things get pretty complicated. We've been working on that as part of the year. That involves lighting for those models, how the materials look, do you do holograms? How do they appear in the UI? Do they appear in a screen? Do they appear over a screen? Do they appear without a screen? If you can have hologram UIs of floating letters, ships and stuff like that. There's all kinds of exciting rendering tech that we've been involved in with that. I think the first thing we've seen it in is the inventory. That was really, really quite complicated. We show 50 models at a time, and each of those individual models has its own scene, its own camera, it has three lights. So there's just thousands and thousands of little objects we have to manage all the time. So that's what we've been doing over the past year. So as a core technology group, we develop that stuff ready for the gameplay teams to use. So I'm hoping that we're in a space that they start using it, and then they'll develop their UIs, and hopefully the players will get to see that in a couple of releases' time after that, because we're right at the start of that chain. So it's now in a state that the gameplay teams start to use it, and then they come back to me and say, hey, it's not quite working correctly yet, and then I fix a few things, and then they can start developing their features with those kind of things.

As well as the 3D, actually, we've put a lot more time into reusable pieces as well. So maybe when Bone did his talk before, I know Zane showed some restylable stuff, which is really cool. But the actual process to do that was pretty complicated. So if you wanted a button, you had to make it from a square object and put some text on and all that kind of thing. And we've got a system now which means we can package those things up. So if somebody wants a button, instead of having to build it from other pieces, they can just drag a button into their scene and change the name on it. So we've added quite a few bits like that. It doesn't make a massive difference to what people see at the end of the process, but for less technical people, game designers and artists, it's a lot easier for them to just build a screen out of pieces. And also, all those pieces are restylable as well. So for example, at the moment, we've just added a Crusader style to the game. So if someone's already set up a screen, we can just turn it to Crusader and it will change colors and some of the graphics will change and so on, which is kind of cool.

Well, I think clean up is probably a bit overdoing it, but I definitely get a lot of calls during the release phase of saying, hey, I've made all this and it's not working quite how I expect it to. Now, is that because they've implemented it wrong or is it because maybe my stuff's wrong or the core tech is wrong? Trying to help them debug what's going on and find out what the problem is. I think that that's grown, but that's a really good thing. I can only see that as a really positive step because we get lots and lots of people contributing now. And being a support role is a lot easier than what we used to do, which was having a really compressed role at the end of the release cycle, which is all the gameplay teams have finished their feature and just go, it just needs UI now. Can you just make the UI for it? That was really tricky because you had to really get involved in lots of other people's work and understand what they want, then design it and write the tech and the UI for it. These would all be completely individual systems. You'd be working on vehicles one week and then you'd be working on the mobiGlas the next week and then you'd be working on the front end. That was really difficult. You'd get these jack of all trades, master of none, because we were diving in and out, but they were all equally very important at the same time. But now that that's spread out, offering support is a lot easier because I'm supporting them in a role where I know the tech and I can give advice about the stuff that's going on. And I also can hopefully write tools to help them debug their own stuff as well.

One thing that's interesting is with AI in games, you get this thing called emergent behavior where the game starts doing stuff that you didn't expect. We almost have a similar thing with the UI tool where it's not AI that's doing it, but we'll decide to do a particular thing and we hand it over to people to actually use and they come up with all these crazy ways of using it that we never expected. They're usually doing really cool stuff, but I suppose there's two sides to it. Sometimes we'll get something that works really well, and then we'll look at the file that's been made to construct it, and it's like spaghetti. There's just connections everywhere. So we sometimes go in and kind of tidy that up a bit and let them know, here's another way you could do it that's going to be a bit more efficient, that kind of thing. And other times, it's like, there is no better way to do this. So Bone, for example, might go away and write a new system that does that thing that they want to do or do it in a better way. That's where a lot of the support lies as well.

OK, so it's progressing. It's a really big system. What we want to do with it and what we've designed in the background, which we're not showing anybody yet, is this really big system that kind of combines the radar, the star map, potentially some of the AR markers, and also the interior map. We think really that should be one thing, essentially. Chris Roberts has always had this idea that he wants to be able to start from the radar and zoom out from that. Then you can see the star map and you can zoom out and see the galactic view, essentially. We've been writing around that idea. We've got this really massive design that ties all those systems together, which is pretty cool. We started writing code systems to support that. One thing that's tricky with it, there's all this data from all over the game. So we've got the planets and stuff. We've got the radar data. We've got even things like missiles. We've got to somehow tie that together in a way that we can put into one place. So we spent quite a lot of time working on the tracking system for all that stuff. So that's all good now. And we're kind of at the stage where we want to start building the real thing. So the challenge is spending the time to fit all that stuff together. So that's something that we've been kind of ticking along in the background, but we're going to start ramping up on that very soon, I think. Couldn't give you a release date, but we're really keen to get it going and it's moving.

Well, we are working. I literally was working on it this morning.

Because these are UI guys. They're not building the mining. The team who is doing the mining can continue to work on mining while these guys work on their other stuff. It's not an either-or proposition; all the teams don't just work on all the features.

Yeah, that's true. The main way that we support other teams at the moment, we're kind of the start and the end of the process. So Bone's team makes the initial bits for people to make their feature from, I guess. And then UI team will sometimes lend support on the visual side as well. So at the moment, we're working on some refueling stuff. So EU PU have been working on that. They've made this system that works really well. They've done UI that is OK. It doesn't look great. But they're not a team full of UI designers and artists. So what we'll do is pass some of the stuff over to our visual guys to upgrade it and make it look cool and Star Citizen. The end product is really good. In general, it's the code side that's involved with things like the star map.

I will say it's working. Those who have been following the project for a while, going back to 2014, 2015, 2016 days, might remember a familiar refrain: this feature is waiting on UI. That was such a common refrain back in those days because of what David just said earlier about how you'd get to the end of the feature, then it's like, okay, slap a UI on it. It was very clear that UI being downstream of everything was not the solution for a project as broad as Star Citizen. So not to keep harping on the success of Building Blocks, but these new refinements to process that you guys have made and the new tools that you've provided have definitely sped up the process. That said, like any team on Star Citizen, there are only so many things we can work on at any given time. You have to prioritize and stuff like that. So not everything can be worked on all at once.

I was at this point of that, when everyone was waiting for UI, and this is why Chris correctly pivoted the team at that point and said, this isn't going to work, because we want to grow how many features we're going to develop. Unless we just magic out of thin air another 20 UI guys, it's just not going to grow. It just turns into a great big bottleneck. So that is why around 2017, I think it was, we did change the mechanism, so you develop the UI and go wide and spread it across a lot of these teams. I want to say put the onus back on them, because when any trouble comes in, it's not just developing it, it's at release time. Inevitably, there's a few problems with any given feature, but if you have 10 features and each of them have a few UI bugs in them and they all have to be done by one UI team, it becomes a real bottleneck. So it was really good to go wide across the company and each of those guys then get the ownership of their particular feature from start to finish and not just go, it's done, just needs UI. We're just waiting on UI guys. It's like, you are the UI guys now, with nice fancy graphics and animations from the art and design departments.