Billboard(Z) not really acting like a billboard

#1 Can you show some screenshots of the problem? I'm having a little trouble visualizing what you mean :(

#2 Ben,

I hope this clears up what I'm talking about. The image only expresses one axis, but you can easily expand to 3 axes in your mind.

A CORRECT bilboard should rotate to always have its normal facing the cameras LOCATION.

In Torque the billboard rotates based on the cameras ROTATION, not its LOCATION.


-Ed

editted image to be as clear as possible.

#3 Ahhhh.... I see. That diagram really helps!

I think (though I am not a GG employee) that such a patch, if it were short and simple, would make its way in. Perhaps a GGer can comment here, on whether Torque's way can be viewed as correct?

To me it seems like a simple gotcha. Go, fix, and submit as a patch. If they don't like it, then post it as a resource, cuz I'd use it.

#4 Its on my list. I'm currently looking at some minor mods for water:

1. Submitted patch to help fix 'in water while parallel to water block' issue.
2. Working on making water adjustable by 1 or 2 meter increments instead of the current 32 (or 64 for blocks > 128x/y)


I'll look at the fix for this next.

-Ed

#5 Hey, did this ever get fixed? I was adding X and Y billboards when I came across this.

I'm running 1.3, and it's still rotating parallel to camera instead of towards camera.

The offending code is in tsMesh.cc

void forceFaceCameraZAxis()
{
   MatrixF mat;
   dglGetModelview(&mat);
   Point3F z;
   mat.getColumn(2,&z); // this is where the z-axis goes, keep it here but reset x and y
   Point3F x,y;
   if (mFabs(z.y) < 0.99f)
   {
      // mCross(Point3F(0,1,0),tAxis,&x);
      x.set(z.z,0,-z.x);
      x.normalize();
      mCross(z,x,&y);
   }
   else
   {
      // mCross(z,Point3F(1,0,0),&y);
      y.set(0,z.z,-z.y);
      y.normalize();
      mCross(y,z,&x);
   }
   mat.setColumn(0,x);
   mat.setColumn(1,y);
   mat.setColumn(2,z);
   dglLoadMatrix(&mat);
   if (TSShapeInstance::smRenderData.objectScale)
      glScalef(TSShapeInstance::smRenderData.objectScale->x,TSShapeInstance::smRenderData.objectScale->y,TSShapeInstance::smRenderData.objectScale->z);
}

#6 "did this ever get fixed?... it's still rotating parallel to camera instead of towards camera."

You answered your own question.

#7 I was just wondering if it had been fixed in HEAD. I'm not using the current codebase.

I just checked my latest cvs checkout and the code looks the same.

I'm rusty on my 3d math, so I was hoping someone would see this and just toss up the code to fix it. I'll test it if someone puts it up.

Thanks

#8 Go grab the most recent HEAD... Do a dif (WinDIF) to see if the files have changed... If not then it hasnt been changed... The change isn't that difficult to do (I imagine) ... submit the change here and Ben Garney will take a look at it.

#9 It has not yet been fixed in the public repository, but we have it in the bug tracker and it should get dealt with for RC2.

As it happens, there are problems with both approaches - the camera matrix hack we use (which doesn't bug on translation, but has issues with rotation) and a distance-vector approach (which is fine on rotation but breaks on translation). I am currently unaware of a fast "always right" algorithm, so what will likely happen is we'll add a keyword you can put in the name of billboarded meshes to select one approach vs. the other.

#10 Well here's what I did for my code:

I want the ability to be able to billboard a whole shape outside of the dts export, so I added a field to ShapeBase which is a billboard type enum (None, All, X, Y, or Z). I added the channels of flags to work their way up to TSMesh's render methods, which is kinda gross.

It just kinda seems like the billboarding shouldn't happen there. You not have a camera or a world object position (the two things you need to calc the true billboard rotations) inside TSMesh.

I'm still learning the architecture here, but there's a lot of code to read through.

Basically I need a spot to move the billboarding code to that satisfies the following requirements:

1) SceneState is available, so that we can get the camera transform
2) Object world transform is available
3) Mesh flags are available (to bb from dts export)
4) ShapeBaseData is available (to bb from script)

Once I can find the right spot to perform the rotations, I am going to apply the math found here

#11 OK so I implemented some scriptable object-level real billboarding separate from the mesh-level billboards. The reason for this is that at the mesh-level render, there is no knowledge of world space or camera position, and I don't really need mesh level billboards so it's ok for me to just leave that as-is.

I added a field "billboard" to ShapeBaseData with the possible values:
0 = None
1 = All
2 = X
3 = Y
4 = Z

I made the field available to the console as well as added a 3 bit int to pack and unpack it.

I added the field mBillboard to ShapeBase which is default 0 and gets assigned from the datablock.

Then I added this method to ShapeBase.cc, which was copied from some fx class:

void ShapeBase::prepModelView(SceneState* state)
{
   if( mBillboard == 1 ) {
      Point3F targetVector = state->getCameraPosition() - getRenderPosition();
      targetVector.normalize();
      MatrixF explOrient = MathUtils::createOrientFromDir( targetVector );
      explOrient.setPosition( getRenderPosition() );
      dglMultMatrix( &explOrient ); 
   } else {
	  //if non-billboarded, just use the normal transform
      dglMultMatrix( &getRenderTransform() );
   }
}

which obviously only does a standard billboard. I'll implement x,y and z later today and post it.
You will also need to declare this method in ShapeBase.h.
Then the only other change is in ShapeBase::renderImage, replace

PROFILE_START(ShapeBaseRenderPrimary);
   if (mShapeInstance && DetailManager::selectCurrentDetail(mShapeInstance)) {
      glPushMatrix();
      dglMultMatrix( &getRenderTransform() );
      glScalef(mObjScale.x,mObjScale.y,mObjScale.z);

with

PROFILE_START(ShapeBaseRenderPrimary);
   if (mShapeInstance && DetailManager::selectCurrentDetail(mShapeInstance)) {
      glPushMatrix();
      prepModelView(state); //Perform any billboarding, etc
      glScalef(mObjScale.x,mObjScale.y,mObjScale.z);

This worked for me. Real billboards :) FINALLY!

#12 I added X,Y and Z billboarding in and shifted a few things around for myself.

First of all I renamed prepModelView to billboardModelView and changed the sig alltogether. Then I moved it down to SceneObject, so that any scene object in the game could use this billboarding.

Here's the new method:

#include "math/mathUtils.h"

void SceneObject::billboardModelView(Point3F cameraPos, S32 bbType)
{
	if (bbType > 0) {
		F32 mulX = 1.0;
		F32 mulY = 1.0;
		F32 mulZ = 1.0;
		switch(bbType) {
			case BillboardXAxis: mulX = 0.0; break;
			case BillboardYAxis: mulY = 0.0; break;
			case BillboardZAxis: mulZ = 0.0; break;
		}
		Point3F targetVector = cameraPos - getRenderPosition();
		targetVector.normalize();
		targetVector.x *= mulX;
		targetVector.y *= mulY;
		targetVector.z *= mulZ;
		MatrixF explOrient = MathUtils::createOrientFromDir( targetVector );
		explOrient.setPosition( getRenderPosition() );
		dglMultMatrix( &explOrient ); 
	} else {
		//if non-billboarded, just use the normal transform
		dglMultMatrix( &getRenderTransform() );
	}
}

I removed SceneState from the signature since this is now in the sim tree and no scenestate refs exist there.

For this to work, you will also need the following inside class SceneObject in sceneObject.h:

public:
   enum {
	  BillboardNone = 0,
	  BillboardAll = 1,
	  BillboardXAxis = 2,
	  BillboardYAxis = 3,
	  BillboardZAxis = 4
   };

protected:
       /// Transforms current ModelView to either billboard or standard render transform
   void SceneObject::billboardModelView(Point3F cameraPos, S32 bbType);

After that is done, you just need to substitute billboardModelView for dglMultMatrix(&getRenderTransform()) in any renderObject() method and pass the bbType S32 to any SceneObject that you would like to add billboarding to.

#13 Ok, for a follow up, I'm having a new issue now. I can't figure out if it's bad math on my part or something along the rendering chain causing it. Basically, when using the above billboarding code, no matter what you do, the object will never be rendered with a negative z axis. At the points where the z axis should go negative, the object's orientation just gets flipped 180 degrees to keep z positive.

This isn't a problem at all for Z billboards (Z is constant), and it's not noticeable for regular billboards as pretty much all applications have Z positive all the time anyway, but I'm trying to do Y-Axis billboarding, and I'll explain why later, but it's causing havok with that.

If you can picture an XYZ axis, with Y pointing out away from you, then my goal is to have it so that the object's vector along the Y axis maintains it's orientation, but the object rotates along it always pointing the X/Z at the camera. This is partially working for me with the above code, except for the Z-Flip. It's super weird. As it's approaching the point to flip z, it looks like the object gets smaller. Weird.

Any ideas? Any of you guys that understand the rendering pipeline better than me understand why this might happen??

I'd appreciate any pointers or ideas here. Also I'll try to provide some screenshots of this happening.

Thanks!!

#14 The createOrientFromDir function takes only one vector as an argument. You need at least 2 vectors to define an orientation. If you look at createOrientFromDir in mathUtils.cc, you'll see it always uses "up" (0,0,1) as the second vector. I'm going to go out on a limb here and assume that this is why you can't get a negative z. ;)

I like your orverall approach and the method you chose to integrate the billboarding into SceneObject, but your vector manipulation is going to need some work if you wish to achieve the desired level of flexibility.

#15 Instead of using createOrientFromDir... I think what you need is two cross products. Take the vector to the camera and cross it with the axis from the current object transform matrix which you wish to maintain. Then cross again the axis you wish to maintain with the result of the last cross, and you should have your rotation matrix.

.. I think. That's just off the top of my head but I think it's correct.

adding: The code is going to get a bit more complicated than that, however, since you need to keep track of which vector represents which axis of the final rotation matrix.

#16 Scott, thank you so much for clearing that up. You were right about the createOrientFromDir function.

I'm trying to do what you suggested but I'm still pretty new to linear algebra so I'm struggling to get this together. I'm not sure what you mean about keeping track of each axis's vector. Also I don't understand how to make a matrix from a vector.

I'll post again if I figure this out in the meantime, otherwise, I'd appreciate any help. Thanks

#17 I've been trying to come up with specific functions for billboarding around the different axis.. I believe I have X and Z worked out, however it occurs to me that billboarding an object around its Y axis doesn't make sense.

If we assume that by billboarding we mean that the object always faces the camera, and that in Torque the Y axis is "forward", then billboarding means rotating the Y axis to face the camera. Yes?

Z axis billboarding rotates the Y axis around the Z axis such that Y faces in the direction of the camera, while still remaining perpendicular to Z. X axis billboarding would rotate the Y around X until it faces in the direction of the camera, while still remaining perpendicular to X.

Either way, billboarding means rotating Y to face the camera. So Y axis billboarding would be... Rotating the Y axis around the... Y axis..? Doesn't work.

So you have Z axis billboarding, X axis billboarding, and full billboarding, where fully billboarding rotates the Y axis to directly face the camera. The question here however, is what is "up"? So far we only have one axis, Y. We need another to establish a proper orientation. Does "up" come from the World, the Object, or the Camera?

You could offer the user the option. Essentially allow 3 types of "full billboarding" where "up" is derived from either the World, Object, or Camera. I'm not sure if that's worth it, however. Using either the World or Object "up" might result in strange flipping behavior when the camera is directly opposite of the "up" vector. Using the Camera "up" avoids this problem.

Of course, using the Camera "up" has it's own issues. That being that if you were to look at a billboarded object while "rolling" the camera, the billboarded object would roll with the camera. Still, I think this is the best choice overall. This isn't even an issue in situations where the camera can't roll, as is the case in many games.

Thoughts?

edit: Minor correction. Typed y where I meant x. Doh.

#18

Quote:...Also I don't understand how to make a matrix from a vector...

Without knowing how much you do or do not know in this area, I'll just attempt to explain a few of the basics.

A typical rotation matrix consists of 3 vectors representing the X, Y, and Z axes of the local coordinate system, expressed in the parent coordinate system. If that didn't make too much sense, try this: For a simple Object in a World, the Object's rotation matrix consists of 3 vectors which define the direction of the rotated object's X, Y and Z axes.

Note that in Torque, X represents "right", Y represents "forward", and Z represents "up". (Not all environments are consistent in this regard. OpenGL actually flips the Y and Z, considering negative Z to be "forward" and Y "up", iirc. Fortunately, we don't have to worry about that, since Torque handles that translation for us.)

So. If you were to take an object in a Torque world, and sit it on its back end such that it faces straight up, the Y axis in its rotation matrix would be (0,0,1), or "world up". If its X axis were (1,0,0) "right", then it would have to have a Z axis (0,-1,0) or "straight back". Follow? I hope?

Keep in mind that these axis must always be perpendicular, unless you wish to distort your object.

On to some real examples. Let's take creating a Z axis billboard. Given a VectorF camVec which is a normalized vector pointing from the object to the camera, we can then establish a rotation matrix like so:

edit: New code below. I forgot a step, and it pushed it over the post length limit.

I haven't tested that, although I'm fairly confident I got it right. :P
Oh, and the Right Hand Rule for Cross Products is defined here for those that aren't familiar with it.

Also, this code billboards around the Object's Z axis. This is not billboarding to the World Z axis. If you place your object in the world such that it is upright, it will billboard as if it were billboarded around World Up. Like one might expect of a billboarded tree, for example. However if it is not place upright in the world, it will billboard to it's own "up". I think this approach offers a greater flexibility then forcing billboarding on World Z. However if that's really what you want, just set axisZ to VectorF(0,0,1) instead of pulling it from the current render transform.

#19 Ok I tried that out, and what happens is it creates some sort of non-uniform scaling that scales the object along the y axis out to the camera. So the farther away you are, the larger that axis gets, and it always stays attached to the camera position.

for camVec I just did cameraPos - getRenderPosition();

I understood what you explained up top, I didn't understand the 3 vectors of the rotation matrix before though. I feel like it's an unnecessary number of points for rotation, though. I was trying to use Euler rotation before, which seemed more straightforward, although clearly I didn't know what I was doing there, either :)

Anyway, any idea on what the deal might be with the scaling? Maybe we need to copy that over from the world transform? Or should these be using normalized vectors or something?

Thanks again for all your help

#20 I normalized each every vector and it all looks correct to me now. Is that right? I was kind of shooting in the dark with it because I thought since we were dealing with the object's render transform that you wouldn't want unit vectors but it appears correct that way.

I'll go ahead and finish this for the other 3 modes of billboarding I want, then post the code after it's tested.

Scott - Please correct me if I was wrong in normalizing those, if there is a more correct way of doing this.

Thanks