Fun with BVH Files, in Torque.

WARNING: this blog is rather long and technical, and very heavy on animated gifs! Feel free to cruise through it and just look at the pictures. Or if you just want the code, skip to (HERE). I am including gratuitous explanations in the pages below, in the hopes that it might assist some artists or programmers to have a better understanding of how shapes, skeletons, and sequences work in Torque. Take from it what you will.

This resource is not guaranteed to work, and the user should download it at his own risk. Chris Calef is in no way responsible for anything that might happen as a result of downloading this resource.

----------------------------------------------------------------------------------------

So, anyway, a few weeks ago, an acquaintance dropped a three-letter acronym on me that momentarily changed my life and transformed me into a single-minded, obsessed, fanatically driven introvert. (Okay, so it wasn't that big of a change. My friends didn't really notice.) He said:

"Hey, do you know anything about bvh format?"


I'd had experience with motion capture before and knew there was a lot of data out there -- but I've been busy doing other things for the past couple of years. I have been interested in the engine side of character animation for a long time, though, and wanted to hone my skills a bit in that department anyway.

But as an indie developer, I also just want an easy, cheap or preferably free way to make use of the huge variety of bvh files that are out there on the internet, without buying or even having to learn to use any major art applications.

Call me lazy, I guess, but I'm a coder. I hate using GUIs.

I do also have the impression, though, that even with all the latest tools and techniques, it isn't always easy for artists to convert an animation that was made on one skeleton into a sequence that works with their particular model. The results of failure sometimes aren't pretty.



Well, I'm happy to report, mission accomplished! Some time later, I emerged from my lair with a couple of big shiny new ShapeBase functions that do everything I asked for. Now all I have to do is download a bvh, load up a player, type "$myPlayer.importBVH(coolNewAnimation);" at the console, and voila!


Well, okay, I'm exaggerating just a little. There are a couple of other steps you have to take. But once you get it set up, it's just about that easy.

Just so we're all on the same page, here's what a typical BVH file looks like:


It's a really simple, plain text format that is quite easy to parse. The skeleton is defined at the front of the file, and then the rest of the file is filled with all the frames data. Each line of frames data represents one frame, and all the rotations and translations for all the bodyparts (there can't be any left out) are stuffed into that one line, separated by spaces. Like this:


As you can see, it makes for pretty long lines.

One complication that became immediately evident was the fact that not all the files agreed on whether nodes should have six channels or three. The most efficient and logical method would be to include translation information for the base node, and then only rotations for everything else, since each node has already defined its offset with regard to its parent, and that's all you really need to know.


However, many sources of motion capture data seem to be less than perfectly efficient and logical, and include all six channels for every bodypart. I even found one bvh file that had NINE, adding scale to the end. (It was always "1 1 1", but it was good to know it was there, just in case.)

Also, many files had a lot of float garbage in them (numbers that should have been zero coming out like "-8.88178e-016"). This may not be terminal, depending on how you're scanning the data, but it's just ugly and I wanted it gone before I even started thinking about making these things work with my character.

So, I added another little ShapeBase function: "$myPlayer.cleanupBVH(coolNewAnimation);". What it does is loads up the entire bvh file (in this case "coolNewAnimation.bvh"), changes float garbage to zero, changes channels to three, deletes all the node translation data except for the top node, and then writes it all back out to "coolNewAnimation.new.bvh". It also cuts everything back to a reasonable number of decimal places, like "4.356" instead of "4.3564786755". These files are plain ascii text, and are pretty darn big anyway; I found that just by chopping off that unnecessary resolution, I could drop the file size by a lot.


So, cleaned up, shrunken down and reasonably efficient motion files in hand, the next step to make this work was to get it into a Torque character. For this, I started with a handy character from BrokeAss Games, who was rigged with globally aligned local bodypart orientations and hence was a convenient testing ground. (If you have no idea what that last sentence meant... stick with me, all will soon be made clear.)


This is where the rubber really hits the road, because the main problem people have with using other people's animations (and this is true whether we're talking about dsq files or bvh files or anything else) is the fact that all models do not share the same skeleton. Even though all normal humans may have two arms, two legs and a head, there is no protocol in place to define whether the right upper leg is called RThigh or RUpLeg or right_femur or what. It's entirely up to the artist, and this is the first of our problems. Often this problem alone keeps people from being able to share animations.

It gets much worse than lack of naming conventions, however. Even if we had two skeletons with exactly the same bodyparts, with the same names, defined in exactly the same order, the real stickler is going to come when we look at how the geometry itself is aligned with the world. When an artist is making a character, they might decide that the right upper leg could be represented by a cylinder, which they will then modify and attach to the rest of the body. This cylinder will be created by their modeling application on some initial axis (like, horizontally on the X axis), and they will then rotate it around to the position they want that leg to be in, and attach it to the body.



This is all well and good, but this means that when we want to rotate this bodypart, we have to be aware that it has local axes that differ from the world. For the right upper leg, the direction that the world calls "down", or negative Z in Torque, might actually be positive X.

This problem right here is why you can't just past rotations from one model into another without them looking really stupid. Working around it can be kind of a brain twister. I had to solve this problem once before for the ragdoll pack, though, so I wasn't intimidated this time.



The solution I came up with was to address all of the above problems in one little plain text config file. Often one config file for a model will work with a whole set of related bvh files, but I set it up so you can make one config file per model per animation, to cover all possibilities. You simply name your cfg file after a particular motion (if you have a "frontkick.bvh" to import, then name the config file "frontkick.cfg") and keep it in the same directory as your bvh files for this model. If the system doesn't find a cfg file named after this motion, it then looks for one named "default.cfg" in the same directory.

Default.cfg for Kork came out looking like this:



The first column of numbers is the index of the bones in the bvh skeleton, mapped to the second column which are the corresponding nodes in the dts model. All the rest of the numbers are Euler vectors that represent rotations.

Along the way here I also added a few handy console methods, just to get a better look at my models and sequences. One of them, "$myPlayer.showNodes();" prints out the list of all the nodes in this shape, with their IDs. It makes the mapping part a little easier.

In this case, I had the following skeletons to match up:

KORK

BVH

The reason I started with the BrokeAss model instead of Kork is because when you look at him in ShowTool, you find that in his default position, standing straight up with his arms held straight out, all of his bodyparts have local axes that exactly match the global axes. With Kork, as with most other character models, this is most definitely not the case.

This is only our first problem, however. When it comes to node rotations, there are really two separate issues to deal with. Besides our local node transforms, we also have to take into account the fact that our models may not be starting in the same position, either! My BVH character is defined with his arms down at his sides, but even my friend the BrokeAss Guy starts his life with his arms straight out. This means that even though a Y axis rotation might mean the same thing to both characters, a ninety degree rotation for the BVH guy lifts his arm from down to straight out, while the same rotation on the game character raises his arm from straight out to straight up over his head!

When we get to Kork, he's even worse -- he defaults to "standing holding a weapon" where none of his limbs are aligned with any axis at all.



So, let's take a look at that cfg file for Kork again:

0;0;(-0.0674,-86.4207,-0.0969);(0.0,90.0,0.0);(0.0,-90.0,0.0);(0.0,0.0,0.0);
1;18;(11.7373,-168.9883,-27.6818);(0.0,180.0,0.0);(0.0,90.0,0.0);(0.0,0.0,0.0);
2;19;(0.0,0.0,-44.9645);(0.0,0.0,0.0);(0.0,90.0,0.0);(0.0,0.0,0.0);

After the node mapping, there are four sets of Euler vectors (rotations about X,Y,Z). The first two are for putting the body physically into the position of the bvh character (arms down), and the second two are for dealing with the local rotations of the nodes once we get into this position.



That file's pretty ugly, though, so let's go back to the easier one. Recall that our BrokeAss human has his arms out, and all I need to do is to move his arms down in order to match my bvh position:

10;30;(0.0,90.0,0.0);(0.0,0.0,0.0);(0.0,-90.0,0.0);(0.0,0.0,0.0);
11;31;(0.0,0.0,0.0);(0.0,0.0,0.0);(0.0,-90.0,0.0);(0.0,0.0,0.0);
12;32;(0.0,0.0,0.0);(0.0,0.0,0.0);(0.0,-90.0,0.0);(0.0,0.0,0.0);

How about that? What we're looking at right there is the left shoulder, elbow, and wrist. Since the local axes are all global, and our character has Z up and Y forward, then all I have to do is rotate his shoulder 90 degrees on the Y and it will make his arm point down to the ground instead of out to his left.

This is all good, but what happens to our local axes when we do that? Suddenly our left shoulder, forearm and hand, which used to be pointing toward toward negative X, are now pointing toward negative Z! Oh no! What do we do?

That's where the last two numbers come in. No matter where we end up, it is always going to be possible to rotate our frame of reference back to the global, in two if not in one set of X,Y,Z rotations. (It's possible in one, but for humans it's sometimes much easier to use two, so I included that option. It turned out to be a very good idea, as we'll see shortly when we get back to Kork.) For our BrokeAss human's arms, then, we simply take the reverse of the ninety degree Y rotation we did at the shoulder, and lo and behold, everything works!

For those who are interested, the actual transform process looked like this:

m.identity();              //Start with an identity matrix.
m.mul(matBvhPose); //move this bodypart from its default pose to match the bvh pose.
m.mul(matAxesFix);  //then temporarily move it back to get the local axes matching global.
m.mul(matY);            //then do the bvh rotations
m.mul(matX);           // ...
m.mul(matZ);          // ...
m.mul(matAxesUnfix); //and then rotate by the _inverse_ of that temporary transform,
                              //to take it back out. Voila!

This wasn't too hard with the BrokeAss guy, since all I had to do was rotate his arms, but what about Kork? What are we supposed to do with a guy who not only has a huge mess of local transforms, but then he's standing there holding a crossbow instead of politely keeping his arms out?

Well, this could be nasty, but it turned out it wasn't all that horribly difficult, either. All you need to know is that Torque has a lot of useful quaternion and matrix functions, and my current favorite is called MatrixF::toEuler(). What it means is if you have a matrix representing your current position, and you take its inverse (which brings you back to the global frame of reference), and then you call toEuler() on that inverse, what you end up with is three magic numbers that you can plug right into my config file. Weird starting position, gone!

In code, it gets slightly complicated by the fact that we have to go from Quat16 to QuatF to MatrixF in order to make it work, but that's not so hard:

q16 = kShape->defaultRotations[2];
q16.getQuatF(&myQuat);
myQuat.inverse();
myQuat.setMatrix(&myMatrix);
myEuler = myMatrix.toEuler();

Unfortunately, if you have no starting position at all, what you get is this:


And that's why I'm glad I gave myself two Eulers per operation instead of only one. Now that I'm back to null, it's pretty easy to rotate Kork's bodyparts around with 90s and 180s until he's standing up with his arms down:

0;0;(-0.0674,-86.4207,-0.0969);(0.0,90.0,0.0);(0.0,-90.0,0.0);(0.0,0.0,0.0);
1;18;(11.7373,-168.9883,-27.6818);(0.0,180.0,0.0);(0.0,90.0,0.0);(0.0,0.0,0.0);
2;19;(0.0,0.0,-44.9645);(0.0,0.0,0.0);(0.0,90.0,0.0);(0.0,0.0,0.0);

When trying to visualize all these rotations, it can help to draw yourself a picture (see below, sorry for the sloppy artwork.) This is Kork with his back to us (so we're all facing the same direction), if he was standing with his arms out. Globally the X axis (red) goes to our right, Z (blue) is up, and Y (green) should go into the page.



And that's really the process, in a nutshell. Once all the frames are loaded and suitably processed, my importBVH function simply adds a sequence to our TSShape, sets it up with whatever flags it needs, and adds all of our new rotations and node[0] translations to the shape's nodeTranslations and nodeRotations lists. (If you have a Torque license and want to see the actual code, go (HERE)).

And with that, I believe I will draw this lengthy and perhaps overly detailed blog to a well-deserved conclusion. In the tradition of free motion capture files on the internet and many years of great advice and free stuff for Torque, I'm just going to throw this little project out there gratis as a "thank you" to everybody who's ever helped me figure something out. Have fun, and make awesome games!!