SBe: As you said, my name is Steve Bender. I'm the animation director at Cloud Imperium. And I oversee the creation of animation largely for things like the player and for the AI in the game. I also shoot motion capture, direct motion capture. I work with the different designers and design leads in laying out, working to figure out how something's going to work, what animations need to go in there, how they combine, and things like that. I'm really lucky in that I have a great team, a great animation team that works on this project, so it's not me doing it. I'm working with these talented animators.
SBe: It's the character motions. So things like, for instance, when you're doing those takedowns, your hands in the view, or if you're kicked out to third person or they're seeing you in third person, the third person view of that happening. The victim who ends up being quite unfortunate in that. That's all character animation. As well, our guys will author motion for the view, especially in the case where they're doing particular actions, so in the takedowns and stuff like that. Because one of the things about a first-person game, when you're playing in first person, is, as compared to a third person game, so I'm running around all the time. I know who I am. I know what things feel like, how they're supposed to play. When I jump, I know how far that's supposed to go. When you're in a first person game, the only thing you have is the camera and then your hands. And that's your only real frame of reference. I mean, you can look down and see your feet, but you tend to run into walls, right? So basically all of the personality or type of character that you're playing needs to come through those hands and that camera.
SBe: Absolutely. So for those of our viewers who are well versed in firearms and things like that, if you have a firearm in front of you and you're reloading it... The best way to think of a first person camera, at least how I think of it, is you put glasses on with a box on it and that's your field of view. And that's what you're going to see. You don't see anything down here. There's no peripheral; unless you look down, you don't see it. So if I'm going to reload and I'm reloading down here, so my view box is open and I can look around, if I show exactly that in the view from my eyeballs, I'm not going to see this action. When I reload in real life, I'm not looking down to see this. I can feel where that button is. I go like this, and I pull this mag out, and the way that I hold that mag when it goes back into the pistol is by feel. I could do it in the dark. So I don't need to have it in my view. That's just one simple example from combat; a lot of these different reloads and things like that, you need to kind of cheat them up so that the player can see what is happening.
SBe: It's like, have I finished reloading? You'd have no idea what your actions are. Because the other thing is when you do these things in real life, you understand. Like, I'm sitting in this chair right now. I know my arm is on this chair because I can feel it. But if I can't feel it, how do I know it's there? So it's the same concept. There's lots of things that go on as you're moving through your everyday world that you know where things are. You know your hand is here. It's not just all of a sudden going to turn around and hit you in the head, right? But in a first person view, in a first person game, you don't know where that thing is unless we show you. And the trick is you don't want to be walking around all the time with your hands like this because that's just creepy. Nobody does that.
SBe: Well, the way that we deal with the first-person and third-person connection in Star Citizen requires us to sort of... split the difference would be a great way of describing that, because we don't want to put it here because then it's going to look really weird, every player running around. It's going to look really weird in third person. So we have to kind of go, okay, there. That's about our distance, and that's going to look decent in first and third person.
Jared Huckaby · Host, Creative Content LeadSo every time you're reviewing an animation, you have to find this sweet spot between what works for first person and what works for third person?
SBe: Yes.
SBe: Oh, wow. That's a whole... No, it was not [easy]. Because of how the third person animation... so if you just put a third person animation on a character and you have your camera, if you want your camera coming from the head, and you put a third person animation on a character and you move around, you are going to be seeing this down here. It's not going to look right. But yet we still need to be able to control certain aspects for the player in sort of the player's space. Any move that I make in third person, and I go like this, all of a sudden, in first-person camera, looks like this. It's enormous. So what we had to do is take that third-person animation and then lock it down with sort of like an aim pose, if you will, on top of it, and then add certain actions on top of it, so certain motion on top of it, which we could then craft that sort of control on.
SBe: Am I allowed to no comment this question? I just did move out to Manchester. There's been a lot of creative technical choices that we've had to make to be able to get other technical choices to work.
Jared Huckaby · Host, Creative Content LeadIt's a very diplomatic way of saying no feature exists in a vacuum. You can make this work on its own, you can make this work on its own, you can make this work on its own, and then it's when you combine them that bugs are born, and that troubles are born, and that backlogs are born. But ultimately, it's part of the DNA of our project. That would be the easy way, and that's how many other games have done it. But part of this endeavor we are on, and part of what I really like about working for CR, is he'll always kind of push: okay, so no other game does it that way, but is it because they just haven't figured it out? So we tend to push into things like that, and that camera actually being in the eyes is one of those quintessential Star Citizen things that I really like about the project, even with all the headaches.
SBe: So we were actually talking about this the other day, in that I hate the word AI. And the reason I hate the word AI is because from a code standpoint, an entity goes from point A to point B, and then it goes here, and it goes to point B, and then it goes over there, and it goes to point C or whatever. It can do whatever it wants. It can go down the hallway and immediately turn around, like this, and flip around and go back, because it's an entity. But human beings don't react that way, right? There are decisions that you can make from a code standpoint where you say, my guy is going to strafe to his left. He's holding a rifle. He's going to strafe to his left, and he's going to go up that staircase, right? Anybody who's ever tried to... and if you haven't, get out of your chair right now and go do it. Try and strafe upstairs while pointing something. It doesn't have to be a gun. You can go get a spoon or something. And concentrate on whatever you're pointing at, and try not to fall up the stairs, right? That sort of thing: an entity can do it, a person can't. And the key here is that the animation's job is, in each of the animations, to look like a real person doing something, but then also the animation team's job is to work with the AI programmers and the AI designers to talk about how something that's happening is not what a person's going to do. Like for instance, the other day we had an instance where the pathing code, we noticed, had characters going forward and then they would flip and turn around, come all the way back. And then they would make these hard right 90 degree turns. And the character was doing this and it was coming back, and it looked like one of the most immersion breaking things that has happened. It was like, what the heck? So then we said, okay, well, part of our job in animation is to describe what the AI code is doing in a way that's believable for performance. So why would you go that way and then all of a sudden stop and come directly back? You forgot your keys. So what we do is then we go, okay, well, let's insert an animation there where your character's walking and came back. And when we made that animation and animation programming injected it into the system, all of a sudden what looked like a stupid decision that was being made by the entity looked like a person who forgot his keys and was going back, and you bought it. It was believable.
Jared Huckaby · Host, Creative Content LeadThe AI animation, one of the reasons I really want to get to this topic, is that there are many things that contribute to immersion in a game. And everybody knows one of Star Citizen's big buzzwords is immersion. Having a fantastically drawn environment with a fully realized atmosphere and proper lighting that behaves, those are all insanely important. A soundscape, things echoing the way that you expect them to, voices coming from the direction they're supposed to come, all those things are super important. But when you see non-player characters do that instant 180, unless they're the Queen's Guard in front of Buckingham Palace, it's one of those things that instantly breaks your immersion. So animation is one of those tools that doesn't get the same spotlight that fog and lighting and sound does, but it's absolutely essential to making a place feel alive, to make a place feel believable. I've already seen the YouTube comments while I'm talking, because a lot of folks' perspective is only what's in the PU right now. And of course, we know that such a big focus is on Squadron 42 at the moment. But I want to talk a little bit about what I've seen in Squadron 42: the life cycles, the people who roam the halls, the janitors who sit there and they mop, and then they kind of stop and they think for a second. Oh, my lost keys, I've got to go back and do this. The people in the mess hall who are going down and they get this and they get this and they kind of think for a second: okay, no, I'm going to get this. It's those moments, just those tiny moments of, I don't know what you call them, that animation insert, that consideration or whatever. It's the difference between a ship that seems pristine, oh, that's a nice model, and suddenly now it's a place that's lived in.
SBe: It's about thought, right? So when I talk about that, I try not to use the word AI a lot, and I try and use character or people or Doug Johnson or something like that, right? It's about keeping in mind that the people moving through, the entities moving through the world are supposed to be people. And in fact, we were just talking about it today: someone was using a locker, and the character goes up and the animation stops. And then he opens the locker, and it was like, why is he doing that? Well, because of... No, no, no. I don't need for you to tell me how many frames are there, the technical reason why. Tell me why he just stopped and stared at that locker. Get rid of the pause, right? Let's cut that out. Let's trim that together. Let's make that flow. We had another example where there were people picking up crates and moving them, and we have to have a technical point in the behavior where the AI needs to check where it's going to go next, where it's going to path to next. And where they had placed that point, with the myriad of different fragments that they had that contained animations, was after the guy picked up the box. And so he picked up the box and stood there like this, and then walked off. It was sort of like, once you get over the age of 40, if any of your viewers have experienced that, you have these moments all the time. But we talked about where we could... okay, if it needs to happen there, you could have the box there and sort of look around: where am I supposed to go with this? Or we could shift it earlier. So we come in, we acquire the box, or we know what box we're going to use. We pull out a data pad. And as we're doing this, the AI's pathfinding to where it needs to go. The second it finishes, we put the data pad away, pick the box up and move. So how do you hide those sort of choice points or those pathing issues that the AI need to make? They're not as big of an issue when you have a game that is local, but when you have servers and those times that are happening, you do need something to sort of hide that that answer hasn't come back yet, and it needs to look like a performance. It needs to look like a thinking person.
SBe: Well, one of the things that is really kind of interesting, and you can actually test this right now in the game, and you can do it as players to players if you're using the FOIP system, for instance, is put your helmet on. Put on an opaque helmet. Look at what you're looking at. And what you're going to find out is that you're looking at the person in general, right? He's waving or he's doing this or he's pulling out this weapon or whatever he's doing. The second he takes that helmet off, you're looking at his face. Everything happens right here. So much of our understanding and connection with people happens just in this tiny little area that it becomes exceedingly important, especially in the case of, say, cinematics, the work that the facial team has done on the cinematics or the work that they've done on the marketing stuff. It's also important in the case of the AI. So that's another thing that we're going to be coming on with the facial team: as the body animations, and the way that we've had to shoot them, which is often without a head cam at the same time... the facial animations start coming onto a character. We had a perfect example of that yesterday. There was a character sitting on a bunk bed, and they were looking at a mobiGlas, and their feet were moving a little bit. They weren't kicking them back and forth or anything like that, but they were like... and you're looking at my face right now, I can tell, right? They're doing this. And one of the designers said, well, we need sad, we need happy, we need nervous animations for this bunk bed. Or the person on the bunk bed. And it was like, do you? Because what if I'm like, oh my God, yeah, that's awesome. Or I'm like, oh God, I don't think we're going to make this. Or, oh, you've got to be fucking kidding me. So that's an example of, just from a body animation standpoint, then you put it on that face and you get a sort of deeper performance.
SBe: They're essentially the same exact base. So motion capture and performance capture both use the same mocap suit. They use the same markers. They use the same cameras. They use the same software. We'll bring things in from MotionBuilder and we will be able to previs as well in the engine. It's all the same hardware, same technology. Where performance capture adds a second layer is when you then add the helmet and the... In fact, in motion capture before, we've had the body suit and the helmet on with the camera recording, but because we weren't recording any audio, you could sort of say, well, that's not really performance capture, but you are grabbing the facial animation at the same time as the body. But the whole point of performance capture is that back in the day, what you used to do is you used to capture the body animation, and then you would capture the face and audio somewhere else. Or you would capture the face... The game I worked on ages and ages ago, we did the audio first. So they recorded everything. They had all their lines. We had to put that together into Track View and line it up along with our blockout animations. Then we went and we pantomimed, essentially body-wise, to those audio lines. And then after that was done, in fact, the current technical vice president here wrote this software where we had video, and I would, in the case of this character, mimic the lines that the actor said, also watching the body, and we used that to drive the facial animation. So we had to do it in multiple parts. Whereas in performance capture, you just capture it all at once, and you can master the audio from that audio that you've created that day.
SBe: Well, I got into the suit... So right now in the PU, can I talk about walk cycles as an example? A lot of the characters are walking very static like this because of how the B spaces are authored. And I was like, I can't stand looking at this anymore. So I started going in and capturing long cycles with other motion in it, so characters looking around and things like that. And we started ninja missioning these things in.
Jared Huckaby · Host, Creative Content LeadI'm sorry, what was that word?
SBe: Ninja missioning. It's when it's not on the schedule, but we're doing it anyway, usually after hours or at lunch or something like that, and take this thing and put that in. But we wanted an injured character, and we wanted a tough guy character. So I was performing that day. I had to get into the role and stuff like that. So I was listening to some heavy metal and stuff like this for the tough guy. And then for the injured character, it kind of looked fake that I was limping. So I took a screwdriver and I literally beat myself in the leg with it until it hurt and then walked across the way. I bleed for you, Jared. But yeah, you can come at it from different ways. There's other examples that we were shooting and trying to establish: what is the role of this character, or what are they going to be doing, say, in the Persistent Universe? You're going to be going there and you're going to see the character shopping, right? Getting into that moment, trying to imagine the time I went with my mom to the craft fair, right? And what did that feel like? This is neat. I like this little bobblehead thing you got going on here. And just trying to be honest with it. So you do have to have that performance no matter what, whether it's performance capture, motion capture, combat, things like that.
SBe: It's really hard to be able to understand how important they are without getting feedback from the community. Like, which emotes are they using the most? Which ones don't they like? Which ones do they like? I see dancing a lot and stuff like that. We did do a session a while ago, oh gosh, this was pre-pandemic. We actually had some backers do some emotes with us and stuff like that. And we've talked about being able to perhaps add emotes related to... we have some hard of hearing employees and we've talked about adding in more of those sorts of actions and things like that.
Jared Huckaby · Host, Creative Content LeadASL stuff.
SBe: Yeah. And that you can also then utilize, because we think of language often as this thing that comes out of our mouth, right? But if you don't have the ability to make those sounds, whether it's from a physical inability to do it, or like if I'm underwater, right, and I need to communicate something to you, I'm not going to go, hey Jared! Who says a similar sort of thing can't be true in space? So if I see you at a distance, right, and I need to make some sort of motion or gesture that you can see me from that.
SBe: Oh, God. Well, thinking about the areas that we're... well, no, we have that one. I was going to say, oh, that stinks, but we have that one. I'm really drawing a blank here. I remember we talked... Salsa dancing. Maybe a little merengue, right? I'd love to do the one we talked about. I remember when I first got here, we were talking about emotes: our two-person emotes.
Jared Huckaby · Host, Creative Content LeadThat's what I've been trying to get. I've been trying to lead you to that, the tandem emotes. That's where Star Citizen can take that step beyond.
SBe: You and I can get into the 'verse and tango together.
Jared Huckaby · Host, Creative Content LeadI do merengue, and then it triggers you, and automatically, by proximity, it goes, oh, you're doing merengue B, I'm doing merengue A. High five. I've never seen an actual effective high five, where one person goes up and the hands actually connect. These two-person tandem emotes, that's where I've been slowly trying to...
SBe: Or things like, for instance, a game that I worked on, also a long time ago, had things where you would go up to a character and he would help you up, say, up above an area, something like that, where you accept a request from me. Would you like to salsa? And you say yes, and then we do a little spin or something like this.
Jared Huckaby · Host, Creative Content LeadPeople are calling out that Portal 2 has a high five and stuff. I'm talking about in an MMO though. I'm talking about where you can have hundreds and hundreds of people and it's got to figure out who the right target is amongst a crowd of people, not there's one player and there's another player and it automatically knows who your pair is. I'm talking about in an MMO, hugging. Just a simple ability to hug. I'm a sensitive man. I like to hug.
SBe: Now, hugging is actually a tricky thing in first person. In third person, it's not a problem, because you just look at it, it's just two people hugging. But if you really think about how the camera is working, right, you end up... like, if you really wanted to get up close and personal with what someone is thinking, there's no better way than to have your camera go straight through their head.
SBe: I think I need to answer those backwards. When I was in high school and I wanted to go to art school, I was going to be a graphic designer. And I'd been accepted to college for that. And we went to the Tyler School of Art Portfolio Day and showed my portfolio to a bunch of other colleges while we were there. And my mom was like, we've got to go to this one. There's one in Florida. And I was like, Mom, I'm just beat. I don't want to do any more. She's like, come on, just one last one. So I went there and talked with them, and they said, well, we have this new thing we're doing called computer animation. This would have been the time when everyone was wearing flannel, right? And I was playing Street Fighter II with my friend Brian, who, if he's watching, shout out to Brian. And I said to him, if I get accepted at Ringling, then I would go into doing animation. And they said, why don't you come on down? We got to Florida. I saw two girls walk past me in Florida attire. And yeah, that's how I got into animation.
Jared Huckaby · Host, Creative Content LeadAnd video games, Brian?
SBe: Right, video games. So out of school, the year that I graduated, there were a whole bunch of layoffs in the visual effects industry, and it was tough to get a job. My first job offer that I got, and that was firm, was a company in Portland, Oregon called Image Builder, and they did things like 2D character and special effects for kids' games, stuff like that. My first game was called Tonka Diggin' Rigs. And it had this thing that you strapped onto your keyboard and you turned the wheel like this, and it pushed the buttons and everything like that. That was my first job. And then just bouncing around the industry until one day a recruiter called me and said, have you ever thought about living in Germany? And I was like, who hasn't? No. So, yeah, I went out to a company in Germany. They were in Coburg at the time, which is this really small town in Germany, the birthplace of Prince Albert, actually. I had the interview. They had some after dinner drinks, probably one too many after dinner drinks, and I took the job. And that's how ultimately I ended up sitting here with you.
SBe: The Crysis series. Hands down. There's a lot of similarities that I feel at times here at CIG with that, because I remember we didn't know what we couldn't do. So we just sort of did it, right? And it was always, we want to do this better. The animation in particular, and the cinematics and the performances in particular, on the Crysis series get better as you go: number one, Warhead, two and three. Just the quality of the animation, the little details and stuff like that. Someone said to me just the other day, it was yesterday I think, was asking me about Crysis 3, and they said that they remember playing the game and when they were switching suit modes and reloading, the actual animation of reloading the weapon was different in every suit mode. So in power mode, he'd slam that thing forward. And in speed mode, it was like... and in stealth, it was pulling it back and very quietly pushing it forward. It was sort of like riding it in a way and stuff like that. He felt like that created this experience that really tied him more into the suit mode and what he was trying to do. And that loops right back to what we talked about just like half an hour ago, on the first person view and those hands, and how your hands, in what you do, convey your character and their emotion in the first person view.
Jared Huckaby · Host, Creative Content LeadThose reload animations are exactly the kind of thing I'm talking about. Those things that don't seem super obvious, are not in your face, but those little interstitial moments that make all the difference between something feeling tangible and believable and immersive, if we can say the name.
SBe: I think that's one thing, though, especially in first person, and we have a lot of work still to do on our first person animations and our first person feel, to really dial that down and craft it down to something that feels solid. I like to say that it's not about necessarily what you're doing, it's about the feel of it, especially in the camera. The way that the camera moves, if you're a game that does that sort of thing, can really make an effect on how the player perceives what's going on around them.