Star Citizen: Reverse the Verse LIVE - The Sounds of Space

So we try and be as systemic as possible. And actually, we did some research into sonic booms. So if you're stationary and an aircraft is flying faster than or at the speed of sound, you perceive it as a boom as it passes by you. But actually, that noise is just wafting over the entire surface. So it'd be like a constant scream or loud noise. So do we try and build a system that would calculate air density, speed of the aircraft, and try and model it accurately? It goes a bit above and beyond what we can do in the audio department, but we can certainly do implied stuff. So we have context like: there is a ship in atmosphere, the atmosphere is of X density, it's going at X speed. When it passes by you, we can create an implied sonic boom. We have a very similar system with flybys, and some really great videos of people flying past each other as close as possible. You hear these one-shots and the design of those is awesome. They're really impactful. But Josh has been doing a lot of the stuff with IFCS 2 and a lot of ship stuff, so I don't know if you've got anything to add.

I can imagine it would be something that we could build into the pass-by system that's already there and just swap these assets in when we're in atmosphere rather than building a whole new system for it. Pretty much could do it within that. It would just require some time to actually make the assets for it. It is certainly something that we want to do because it's that quintessential atmospheric flight experience that would really separate it from space.

Of course, it would have to be different depending on which moon or planet you're on. Some moons obviously have no atmosphere at all. Some moons have completely breathable atmospheres. Same with planets. It would be an interesting challenge not just to design a sonic boom system, but to design a sonic boom system that is different and signature to every environment.

And how would that sound? Because sound travels faster through denser mediums. Sound travels like two or three times faster in water than it does in air. So if we're dealing with different densities of atmosphere, how would that sound? Would it propagate for less distance if there's a thinner medium for it to go through?

Depends on the temperature as well, doesn't it?

Yeah, well, that's the thing. We've got this whole universe to put sound to. And it's like, how does it sound? We don't know because we've not been able to recreate those test environments. But we do our best to make it sound cool as much as possible. With the sonic boom stuff, though, I think that would be pretty cool as gameplay feedback. So say on the weapon side, if you hear a sonic boom from an aircraft or a spaceship moving, that would give you an audio indicator that it's moving at a certain speed. So then you can say, well, I know sonically I can engage with that with this weapon or not, or know how fast they're moving kind of thing. So that'd be cool.

It's something that we've developed a system for on the audio side called the audio environment feedback component. Basically, that will handle pretty much every kind of instance where a ship or even the player will come into contact with any kind of particles or weather, that kind of stuff in the future. And that is the first use case for it: flying through the debris of a ship that's been blown up, or an asteroid field, all that kind of stuff. That'll be completely systemic.

And I think one thing I'd like to hear is, if you power down your ship and you're basically just in this metal husk just floating through space, you'd hear all the little tinks of little space debris hitting you, and it'd make you feel really vulnerable. Watching some of the interviews of the astronauts on the International Space Station, they'd say they'll be sleeping, and then they'll hear like a clang. And that's something going like thousands of miles an hour just hitting that space station, like right next to their head. It's a small thing, but it can be terrifying, I think.

Mine is probably the Battlestar Galactica remake, the 2004 one. That's probably my favourite sci-fi series, and I really love the sound design in that. That's got to be mine.

I think I'd say the more recent Star Trek films have some really wicked-sounding spaceship sounds. The third one, is it? Beyond. They do this chase through the city and the ship sounds are awesome. Yeah, they're nice.

For me, Star Wars was a big one. That was just ace. Ben Burtt is the sound designer on that, and he set the precedent for so many sci-fi sounds. A lightsaber, that was awesome. And that was him accidentally walking past a TV with a microphone into a recorder and getting the static, and the interaction of the tube in there combined with two old film projectors getting into sync when you switch them over. All the way to the lasers and then obviously to the TIE fighter. I think that was like a processed pig, I think.

The TIE fighter was an elephant, I think.

It was an elephant, yeah. And obviously all the stuff in there. And then more recently, in the Star Wars films, they kind of got inspiration from a lot of the combustion engines from trucks and stuff like that. So Star Wars is a big one for me. But I really liked Gravity, because that was a great reference of how do you do sound in space without doing the Hollywood rule of cool. And that was all the textile kind of stuff, you're touching stuff and the sound is transmitting through solids and through your suit. And that was awesome.

It's the same reason I like Battlestar. It's not Hollywood sci-fi, but there's still sound in space, but it's very appropriate to what you'd expect a slightly stylized but realistic version of that to sound like.

Yeah, that's a big one for us. So we're in the process of trying to fully flesh out an in-lore sound simulator component module. And we're trying to work out how that fits in with gameplay, because we don't want to give everyone super hearing all the time based on this new tech. But the basic principle is you've got a sound simulator component on your ship. And that will basically, in space where there's no atmosphere, where sound cannot transmit, propagate sound to you as if you were in atmosphere. So how would you hear it, although you can't. And then when you get out of your ship, you may have a light version of that component on your suit. So again, you can hear in space, but only to a certain distance. And then when you get down to atmosphere on a planet, it can also operate, but it might be more beneficial to take your helmet off and just hear the natural sounds as they propagate. But a big thing is system degradation. So if you can target a component and you can target someone's sound simulator and destroy it, they'll be flying around. You've got all these sensors, you've got the visual, you've got the audio going on, you know where everything is positionally, especially if you're playing in 5.1 surround, so you know if someone's behind you or not. And then if that system just cuts out, even just momentarily, you kind of lose your bearings on where everything is. And if it dies completely, then you get to the whole husk just flying through space. All you hear is the sounds of your ship creaking, all that kind of stuff, and then just the impact sounds. And then you feel vulnerable. That would be ace.

That split-second moment where it would cut out has the potential to be so cinematic and jarring of a moment, when you suddenly lose all of your senses basically for two or three seconds. That could be such a cool thing to have.

The stereotypical thing, you see it in films all the time, where people have radio comms back and forth and they get too far away and it all drops out. But imagine that for your entire audio experience. That would be awesome. And it also means that you have to maintain it. So just through negligence, you could not maintain your ship, and then everything starts sounding not as it should and dropping out.

So I have to do my duty here and say that that's not us guaranteeing that that's a system that will go into the game. We're actively working now. It's something that we want to investigate and pursue.

I don't know if we can talk much about Atmos specifically, because that's, again, I think above our pay grade really. But we have been investing a lot recently in surround, in designing stuff in surround and supporting the mix in a surround sound context. We really want to make that a better experience for people that are using 5.1 and 7.1 and hopefully above. I don't know about Atmos though.

Atmos is crazy. I think there's an Atmos system for home now where there are two speakers behind and they reflect off the ceiling. That's crazy.

The whole surround sound experience is something that we've been focusing on a lot for the updated flight model. Trying to get this thing of the all-enveloping sound of being within the ship, rather than just hearing sounds from the thrusters themselves. Hopefully, all going to plan, you should hear a big improvement in that in 3.5.

Way more positional. Because we author a lot of content in stereo in our sound booths. And then we've been making a concerted effort at referencing and authoring more 5.1 stuff. The big thing was, when we were playing in 5.1, all these cool weapon sounds specifically that were in front of you, that sounded awesome, all of a sudden they were behind you in the surrounds, because that's where they physically are in the ship. That's where they would be. That's the closest speaker for them to emit from. We're like, all right, let's try and rein that in, so we don't want all the cool sounds behind you, otherwise you'd be playing like this.

It was Darren. That's a big bugbear. I have a big bugbear where you get these cinematics and they sound nothing like the game, so we try and create synergy wherever we can. But when you're designing sounds for a trailer format and it's linear, there's certain things you can get away with. A big thing is leading up to an event, so the ramp-up to something cool happening. You have way more flexibility in linear media than you do in game. In game you get a call that says this thing happened now, so you can't really lead up to it, unless you want to delay the event, and that has huge knock-ons and stuff. That sound was designed with a certain scope of quantum travel happening, right? The initial one. But now you can do way shorter jumps and way, way longer stuff. So it's definitely an area we want to go back to and bolster with more content.

Because the longest jump you can do at the moment is something like 20 minutes, I think. And we had to put in a quick fix for that too. We got a lot of complaints about people being blasted for 20 minutes.

We have to look at these things and try and do them as efficiently as possible. Now, I don't think we'd ever put sound on physicalized raindrops, but we can take a data set and say: is it raining? Yes. Is it light, medium or heavy? What surface is it interacting with? Glass, metal? There's lots of metal in our game.

That sort of approach of trying to play a single drop sound for every raindrop that hits the window probably isn't even the best way of doing it, to be honest, because there's a certain nostalgic, emotional tone to those, like sitting in the car when it's raining and you hear the rain hitting the windscreen. If you're just using individual sounds of single raindrops, you couldn't capture that feeling of being sheltered from the storm or whatever, which just one really, really good recording that you play with some logic behind it could do.

But it is definitely something we're going to do. It's a great scenario: imagine you're in an outpost and it's all powered up, it's all good, there's a storm coming, you've just made it in time, and then your power module fails, and you just hear all the debris batter up against the side of it and rain on the top. That'd be ace.

I think we've each got one of those. I think the hardest sound I ever designed for the project was when I very first started, all those years ago. I went to the audio director and I was like, "I really suck at UI sounds." He's like, "Great, design all the new mobiGlas stuff." I was like, all right. So I spent an entire week just generating source. A typical process when you're sound designing is you get all your audio plugins and you get all your source material in. For this, I actually thought I'd be clever and use glass samples, for mobiGlas, because I'm clever like that. And you just throw loads of content through these plugin chains and you change them as you go along: this works well, this doesn't work well, generate loads of assets. And then from that, with UI, you have to try and make a language. This is a positive action. This is a negative action. This is for movement. This is for a static button press. So I think all in all, it took me about three weeks to get a first iteration. And that's with me thinking the audio director was going to be criticizing my every move, because I thought I sucked at it. It went all right and then improved, gradually moved on and on.

How about you, Josh?

It's not really any specific sound. The most challenging thing for me has been, in the new flight model, trying to capture the feeling of weight in the ships. When you accelerate at max thrust and then afterburn, you need to really feel like that's a big event and you've got all this power behind you. There's no one sound, it's just sculpting that whole experience. There's been a lot of trial and error. Just balancing all the different moving parts.

Are you happy with where it's at now, or are you still working on it?

It's getting there. I'm excited for people to hear it.

It should be a huge improvement over IFCS 1. And Barney?

I think the thing that I have spent a long time on, and it's been a lot of trial and error and we really haven't got right yet, is the sound of distant weapon fire, when people are on planets and things like that and they're firing their guns at you from hundreds of meters away. It's a really difficult sound to get right. I thought it'd be as easy as recording a distant gun and then using that, but it really isn't. There's a lot more consideration to the sound than just using what you would think is the right sound. Making it sound organic and making it sound real is quite difficult.

Because the complexity of that is: what environment is the sound originating in? What is the distance between you and it, and the medium? And then what environment are you in, listening to that sound? So if someone's got a sniper rifle in a desert, and then you're in a canyon. And it's the initial sound of the fire versus the whizz-by of any ammunition flying by you. So there's a lot to consider there.

Yeah, we could talk about it in different contexts, really.

So looking at the cave example you've given there, we can currently calculate the size of space that you're in by having raycasts emit from the player, and they'll intersect with geometry, and the read from that will say: are you inside? Are you not inside? What size of space are you inside? Are you half inside a space? And based on the data that comes back from that, we can change the reverbs that are applied to certain sounds. And we've not had caves before, so that's going to be a new venture. And that'll be fun to play with, because at the moment we don't honor the propagation of sound through a medium with regards to the speed of sound. So if a sound happens at distance, you hear it at the point it emits, for gameplay purposes. But if you're in a cave and we have dynamic reverb and delay, that there implies sound traveling over time.

So is the question, if I send a message now or I try to open a channel, at the point I'm talking, no one will intercept it until a delayed period of time based on distance?

Imagine once the Stanton system's complete, and you're at one farthest end of Stanton and the person that you're trying to contact is at the other farthest end. Would it still be a one-to-one, I talk, they hear, or are we discussing some kind of delay occurring? I was just watching an episode of The Expanse the other night where they were communicating back from Earth and the Moon and they were trying to have a conversation. They were stepping all over each other, that kind of thing.

Well, in theory, that is definitely possible. But would we want to have that experience in the game? Because that'd be very frustrating, right? If you're trying to have real-time conversations, everyone would just jump on a third party to be able to communicate. But the idea of being able to leave a message for someone to find, that would be wicked. So you can record a message and put it on, say, a probe or something, and then go, there you go, and someone can come across this recording. That'd be cool. Message in a bottle. I don't know if that as a feature is going to come online in regards to distance affecting comms. But what we could do is attenuate the signal based on distance. So introduce more static and more dropouts if you're far away, or based on what quality of equipment you're operating with. So there's loads of stuff we can do aesthetically, but with regards to the gameplay side, do we need direct communication? Yes or no, that's not for us to decide, unfortunately.

Yes.

We've actually been paying a lot of attention to various other games on the market that have much more complex audio options than we do, and we really want to improve that and make it a much more customisable experience.

I think I'd always be a bit reluctant personally to give an ambient slider that's separate to Foley and stuff, because it could give you a gameplay advantage if you can just turn down all of the ambient sound and hear all of the Foley and the footsteps and gunshots and stuff loud and clear. It's more up to us to just keep on making the mix better so people don't feel like they need to have that.

And one idea we've floated is creating mix profiles based on context in the game. So say if you're in a multi-crew ship, a pilot would necessarily need to hear different things to what a turret gunner would need to listen to. The information input they need is different, same as an engineer. So in the future, there may be a possibility to create context-specific mixes where, in this context, you hear X, Y and Z, but in this other context, you hear A, B, C. And then maybe we can make them selectable, or maybe a custom context mix snapshot. We want to give the players as much flexibility as possible, but obviously we want to make sure that we're not skewing gameplay one way or another.

Well, we had this a while back. Some of the standard emotes had some dialogue assets to them. But with VoIP and FOIP coming online, you've then got a dialogue asset or vocal asset you can trigger with emotes, as well as hearing your own voice. So we need to really define: what is your voice in Star Citizen? Is it you? Is it this character? Are you the character? Because when we look at the breathing system, we were saying, well, it's a bit strange, because we're imposing this breathing system on the player for audio feedback, so you know what your stamina is doing or what environment you're in, what context and stuff. But you're also now going to be able to talk down your microphone, and your character could be gasping for breath, and then you go, "Hey, man, pretty chill. How's it going?" So there's a break in continuity there. So with the movement on the VoIP and FOIP stuff, I don't know if we'll be looking into putting audio to emotes, because you'll be able to generate your own.

The main issue with that is if it's on a server and you're broadcasting to multiple people, and then you start broadcasting copyrighted material, then we would be facilitating a breach of copyright. So there's a huge legal problem there. Now, if it was client-side only and you owned the rights to everything and it was just you in that one experience, then there isn't much problem there. But it's when you start broadcasting to other people.

Although, admittedly, people can switch their audio inputs on their computer and do that with VoIP already.

Absolutely, but we don't want to promote it and facilitate it as much as possible.

It's one thing to have a system not designed for it that's being exploited, versus having a system that we've implicitly put in there. So that's not to say that that's a hard no on it. It's just that at this point, I imagine there are much greater priorities in place.

It'd be really cool to share as much audio as possible, but there are certain things we have to abide by.

Yep. We do have that on the Gladius, because there are animations for the flight ready and all that on that ship. If there's an animation for it, then we would do sound for it.

I would hugely welcome that, because at the moment you get in your ship, power it on and you just go "boop". I'd really like a more engaged start-up process. So basically, if it's in the game and it does something, and even if it doesn't do something, then we're going to put sound to it. One cool thing we can do: you can get a completely static environment with nothing going on and you can put loads of implied sounds in there. Like you could put the neighbors next door having an argument, but you never see them. They don't really exist. We just created this. But then if you're getting into your ship and you're doing your flight checks and getting ready to head off, then every animation will have bespoke audio for it. I actually looked at setting up a recording session at the Concorde centre to go in and record all the different switches and stuff. But it never got off the ground. No pun intended on that one.