TL;DRThe item behind him is a bar built for the universe's bars, with props NPC bartenders can use to serve drinks, and the room is an Austin motion capture space with twelve cameras.
So what you're looking at behind me is actually a bar that we've built for our bars in the universe. And if I see if I can slide out of the way there, we have a full layout with things that the NPC, the bartenders, will be able to pick up and use and deliver drinks to other NPCs or to players. If you go and sit at the bar and order a drink, the bartender is going to come up, do a big concoction, hand you a drink.
Now, we can't see it in this shot. You're actually in the new expansion that we've added to the Austin Studio this year. Austin Studio is building out, it's expanding into the area next to it. And this room that you're in, we can't see it again because of the camera, you actually have a motion capture rig all around this room that you're in.
Oh, we do. So we decided to bring 12 of our cameras into our animation pit and set it up so that anytime we need to capture something, whether it's a small transition or we're doing some R&D and blocking out animations, we can quickly capture it, have it off to the designer so they can implement it in the game.
The director usually gives the actor some sort of motivation. Are you just a normal person? Are you happy? Are you gruff and angry? Are you cocky and confident? Are you a Han Solo? Or what kind of a personality are you? The actor thinks about what the direction is and then they make acting choices. And the acting choices, well, you see it, that drives the character. You can see how both of these actors, who are pretty much doing the same performance, are doing different things, even though they're within the same character.
Yes. This right down here is live. This is what was in our software called Motive. This is the capture software we use. So this was live. We come in and we actually go in and we clean up any occlusion or dropped markers, swapped markers, that sort of thing. And when it's clean and it looks good, we then export a skeleton and we retarget to our character.
TL;DROlder motion capture software needed a T-pose to tell body markers apart, since arms at the sides could swap markers; now it isn't required but keeps files clean and marks the animation start.
So what the T-pose did typically in the olden days, back in the olden days of motion capture, you would have to have the actor go into a stiff T-pose because the software needed to be able to recognize the different markers on the actor's body and say, this is an arm, this is the upper arm, this is the lower arm, this is the hand, chest, and so on. When the arms are down to the side, a lot of times those markers can flip and get swapped for markers that may be on the hips or in the chest. So by kind of spreading your limbs out to the side, it helps the system. Nowadays, it's not required, but it's really nice. It's a nice way to just kind of keep your file clean. It also is like a good time marker saying, this is the beginning of the animation.
No, the keys that I'm setting, actually I'll just expand this so you can see what I'm talking about. So here's my curves. These are what the actual animation is that's being applied to the different controls in the rig. Right now, these are set to auto. They're just auto smooth. I actually had them set to stepped, which is like a stair step, and he's just going to rigidly pop from pose to pose, and I didn't want that. I do want that when I was blocking out all my bartender animations. I'm going to put that back to auto.
TL;DRSeated versions of emotes are possible, but for high-fidelity motion they would likely recapture them seated rather than adapt standing ones, since the body moves differently.
There is. We probably want to capture it from a seated position because the way your body moves is a little bit different. In fact, I have some seated animations here I can swap over to, I believe. So here's the facepalm sitting. So we can actually sit here and we can evaluate between these two animations to see the difference. So with the sitting, he kind of leans over and he does kind of the Picard-esque facepalm. Hand doesn't quite make contact, but you can kind of see how he's leaning over in the seated position. If I go to the standing version of it, you can see how the core of his body actually acts and moves and postures differently. So while in theory you could probably play an upper body animation on a sitting set of animations, if you want that really high-end fidelity of motion, you probably want to actually just capture that motion sitting down.
So obviously we have a lot of emotes that can be performed while standing. If we wanted to have versions of those emotes that played while sitting, rather than adapt those, we would probably just recapture them from a sitting perspective to get all that natural nuance that comes from when people sit.
TL;DRThe MotionBuilder control rig combines an FK skeleton (spine, limbs, fingers, neck and head) with a separate IK rig of spheres, and the two switch smoothly without snapping because the rigging is programmatic.
So this control rig in MotionBuilder, and this is why the control rig in MotionBuilder is so powerful, is it has an IK and an FK rig built into it. Everything that you see in yellow, it's pretty much this whole tree hierarchy, this is your FK skeleton. This is your spine. If you think about this, this is kind of like a person, right? Here's his hips, spine, shoulders, down into his hands. I usually color code these fingers so I know that they're fingers, so to speak. Here's his legs right over here. And then right over on this side is his neck and his head. We actually have two neck joints in our character. Over here in the red and blue spheres, I'm going to X-ray him so you can actually see what I'm looking at. So again, here's the FK. Now all the red and blue spheres, these are the IK. And these are kind of like two separate rigs that have a symbiotic relationship with each other. They work with each other really nicely and you can easily go back and forth between IK and FK. A lot of programs, a lot of rigs, you have IK snapping and they have to do all sorts of code and scripting in order to prevent that kind of snapping. But because this is all programmatic, it's all built into the rigging.
TL;DRIK (inverse kinematics) lets an animator drag the end of a joint chain, while FK means rotating each joint one at a time, which makes IK much easier for 3D animation.
IK stands for inverse kinematics and I can show you real simply what that is all about. So FK. So to get FK to work, I'm going to have to rotate my arm up. Let's say he's going to point at something. Then I got to go select my elbow and I got to bend at the elbow and then I got to select the hand and get the hand into position. And that's how I'd have to animate the entire character. And then one day somebody got the idea of being able to create an inverse kinematic chain. And what that does is it allows me to grab the end of a chain of joints and drag it around, making it much, much easier to kind of drag and rotate and point or grab onto something. And I'm old enough to remember what it was like pre-IK. And IK has made animation in 3D so much easier. So much nicer.
It's in the actual capture data. Chances are the marker floated off or got swapped or something along those lines, or maybe there's just a little bit of noise on it.
TL;DRYes, they often capture a performance in separate parts, such as a jump and a separate impact, then stitch them together, for example to avoid actors performing dangerous drops.
Absolutely. A lot of times we will film something specifically in parts because maybe it's impractical or we don't want to kill our actor on stage doing these things. So we say, all right, this one, you're going to jump from this little platform onto the ground. They do it. Then we're going to say, we're going to bring you up a little bit higher. We're going to do the same kind of jump, but when you hit the ground, you're going to have this big impact. So then they do a big impact, right? Then we take the element of jumping down and the element of impact. And then if we have to put a predetermined height on it, we'll raise that character up however high in the air the character needs to be, let them drop, and then we'll stitch those three elements together to make one element of a character jumping off and landing on the floor. And let's say if it's a superhero game and he's jumping off a building, I don't think my actor wants me to have him jump off an actual building to hit the ground.
Typically, we would try to use the emote in its entirety, but I'm going to give a for example. Let's say the data came back, and see how he has this nice swing in his arm like that? And this has happened a lot. The data will come back and he goes shunk and he'll just lock right there. And it looks bad because you don't have that settling. And if I try to motion edit that, it wouldn't feel so good. But there may be another take where he does have a bit of an arm swing that works with it. So we'll kind of piece in and motion edit the ending of one animation onto this to finish it out to make it feel nice.
TL;DRBrewer says ship animations are the most complex because ships keep changing and break character animation, and sitting-console animations use a shared timing metric to reuse chair animations.
Most complicated animation? I have to think about that one. Everything has its own complexities, right? And there's different ways of solving these problems. I'd say probably the most complex ones have to deal with the ships. The way the ships animate, they're super challenging, just super challenging because they're still building the ship half the time. You're getting the ship to animate it and then they're changing things and renaming things and pivots are getting moved and it destroys your animation. So that's really challenging, just getting the ship animated and then getting the character animated on top of that, because characters got to get in the ship, right? And I remember back when I did the Hornet, I believe Hannes did the first Hornet enter maybe? I can't remember if it was him or if it was somebody from CGBot. I went in after I started and touched up on the animation to make it a little bit nicer. And one of the things I did when you hopped in the Hornet was grab ahold of the yoke, which meant when you grab ahold of it, it's animating with the character and you got all this movement in it and everything like that. And it's like, hey, it's great until they decide to change something. It just completely destroys the animations, and things don't line up. And you're like, why is this not lining up?
So it's like a whole other layer there, because you don't just have to worry about the character, but then the ship's animating, and that's where metrics come into play, right? That helps a lot.
Totally. Metrics come into play. One of the challenges that we have right now with our metrics, and again, things that are going to be challenging: one of our animators is doing all the sitting consoles. In order to get the character to sit in the sitting console and look natural, you actually have to pull the chair. Just like if you were to sit at your desk, you don't just sit down at your desk and you're out like this, I'm working at my desk like this. You sit down and you got to scoot up to your desk so then you work comfortably and naturally. Same with getting out of it. So what we need to do is we need to sit that character in the chair and we need to slide that chair forward in order to have the character feel like they're naturally sitting in this workstation. We call them sitting consoles, but just computers. Like you go on the bridge of the Idris and you see all these AI navigating to these seats and sitting down and working these computers. So how do we do that? We got to sit them down. We got to animate the prop to move forward at the same timing. We also don't want to create like 500 chair animations of them sliding forward. So we can't have a unique animation for each sit-down animation. What I had Vanessa do, she's the one working on this, what I had her do is I had her take what's the average time of the character to sit on the chair. That's when the chair starts to move forward. Make every character fit that template. It's basically an animation metric. So hopefully we only have to animate the chair twice, enter and exit.
It saves a lot of time. A lot of time. Memory. It saves a little bit of memory, but it saves memory as well. The more we can reuse an animation, the better.
TL;DRThrottle and joystick movement already exists as programmatic inverse kinematics, where the character's hands follow the controls, though poorly fitted proportions can cause arm stretching.
Yeah, so those are programmatic, right? So when you throttle forward or you're banking or whatever you're doing, dogfighting, what's happening is your character's hands are IKed to that throttle.
Inverse kinematics. Forward kinematics is I grab my arm, I move it, I grab my elbow, I move it, I grab my hand, I move it. That's forward, right? So what we want is we want inverse kinematics, where you grab the end and you pull it. And so I'm grabbing the hand and I'm pulling it out. So what happens is they grab ahold of the yoke and then IK turns on, which now the yoke's in control of the arm. So when you're flying with your joystick at home or your mouse and keyboard or however you're flying, when you're banking and everything, it's saying, hey, I'm going this way. Code says, hey, yoke, you're moving in this direction. And then because your character's hand is IKed to it, it's being pulled with it. So again, back to metrics and making sure things work. If things don't fit and things are not built correct and things are not set up to fit with human proportions and human anatomy, when that thing goes forward, there's a chance that you can IK out, which is joint locking. You see the arm just kind of stretch out like that. Looks really bad and ugly. But yeah, so those are things that we kind of have to watch out for.
TL;DRBrewer says nested animations within a usable, like sitting on the toilet or working at a work zone, are being worked on, with an AI code hitch still being solved.
Absolutely. We're actually working on that right now. There's a little hitch in the AI code, if I'm remembering correctly. Right now, we're having a little bit of trouble. We can get into this usable. We can sit down to sit at the bar or at a mess hall table or on the toilet. And then you can play a series of animations within that. If you want to do something else, for example, a work zone: we have these AIs walking around ships and they want to go up to this, we call them work zones. Really it could be anything, it could be a damaged panel, it could be something that a little AI has to repair or fix. So they walk up, they're gonna enter this, they're gonna look around, they got some fixing animations, but then if they want to do something like pull out our paw tool, which is like a welding tool, now they have to go into a whole different state of, I'm going to grab this usable and I'm going to use this usable and weld. I'm going to holster that usable. Then I'm going to maybe grab another usable like a datapad, pull it out and analyze, put it back, maybe fix something. So there's this whole series of things that the AI will be able to do within this work zone. And we have all these animations. Right now we're just trying to work on the code to allow AI to navigate and do a plethora of things within this usable. So sitting on the toilet, absolutely. Sitting on the toilet with your mobiGlas, nothing better than checking Spacebook on your wrist.
TL;DRShip entry and exit comes in roughly three speeds, like walk, jog and run, with emergency exits not yet shown, so large numbers of variations aren't promised.
Yeah, the emergency, I mean, they're flying. One actor actually broke his ankle or twisted his ankle or did something to his ankle really bad jumping. This guy's a stunt guy too. I think Steve Bender was like, hey, do you really want that pad there? And the guy goes, yeah, that's okay. And he totally missed the pad, off center, and he hit it wrong and he messed up his ankle.
TL;DRMocap uses improvised props such as PVC pipe, plywood, scaffolding and ladders since only the measurements matter, and some shoots have used wire work.
Oh, yeah. Motion capture is very interesting. It's like in a movie, right? You have these prop departments that build all these really cool things for the actors to use on set. We use PVC pipe and duct tape and gaff tape and plywood and everything else imaginable. We use scaffolding for climbing up onto stuff, and ladders that are strapped to it if you've got to go up a ladder, and things like that. If you want to do something on the fly, I want to do this, you go out there and you put out a bucket and a piece of plywood and you hammer it together. You can do a lot with motion capture because you never see it, right? All you care about is the metric. Is it this high? Is it this wide? Is it this far out from this other thing?
And I remember seeing, we released the video for Andy Serkis a while back showing a lot of the Vanduul mocap. And they go all out with some of that stuff. They had him wired up, so when he would jump, he would fly across the room.
TL;DRProduction and management set goals for a release, the team checks existing animations, then plans in detail what and how to shoot, since pre-production avoids capturing unusable footage.
Well, a lot of it's production and whoever the upper echelon of management is at the time for that particular feature. So they'll say, hey, these are our goals that we got to hit by this release or 3.0 or Gamescom or PAX or whatever thing that we're pushing forward, and say, these are what we want. What do we have in our current library of animations? If we have something that fits it, then we'll go and we'll draw from that and we'll make it work. And if it's not there, then we go, okay, these are what we need to shoot. And then we spend a lot of time actually saying, all right, because a lot of times you're not only thinking, what do I have to shoot? You have to think, how am I going to shoot this? If it's something that requires a metric, you have to make sure, okay, we got to go to a cafeteria line and we got to get your tray and you got to put on the thing and slide it down and grab the apple and this and your pie and put it on there and grab the tray and walk over and set it down, because you want this incredible mess hall animation of cool interactivity and stuff like that. So you've got to think, all right, how big is this tray? How high is this thing? What are the elements that design needs to be able to hook this up? We need to enter this particular line. And how does it work from there? We've got a bartender scene coming up and we're talking about how does the bartender function behind the bar? Well, we're going to have these zones, and these zones are going to be shot this way. And so you have this zone for mixed drinks. You're going to do the mixed drinks. You have this zone for taps. You can pour beer into a mug from a tap. Where are the glasses? The glasses are kept under here. We have to figure all this stuff out in advance before we can go to the stage, or we could just capture a bunch of stuff that can't be used. And we don't want to do that.
Pre-production is super important. To the point where, going back to your cockpits, the full-size cockpits that these actors are jumping into for the ship animations, right? All that stuff was measured out and it went to a prop master over at Imaginarium. Him and his team built out these really nice cockpits. I mean, they look like skeletons, like things that you're climbing up onto, but they build all that out and then we show up. And the actors show up and they break their ankle. Or they get on there and they play with it. They figure out how to get in and get out, and then we shoot.