Blender Exporter 0.5 Available

#41 Well, I added your code for the python script. I am still getting the error, but I have not yet modified blender. I did look at the API for the python to blender interface for IPOs. Here is some interesting things I have found:

getCurvecurval(curvepos)
Gets the current value of a curve of the Ipo.

Parameters:
curvepos - the position of the curve in the ipo or the name of the curve
(type=int or string)

Returns:
the current value of the selected curve of the Ipo.
(type=float)

If this is accurate then you can already supply the name of the actual curve. Also:

getCurve(curvename)
Returns the IpoCurve with the given name. The possible values for curvename are R,G,B,SpecR,SpecG,SpecB,MirR,MirG,MirB,Ref,Alpha,Emit,Amb,Spec,Hard,SpTra,Ang,Mode,HaSize,OfsX,OfsY,OfsZ,SizeX,SizeY,SizeZ,TexR,TexG,TexB,DefVar,Col,Nor,Var(Material Ipo) HorR,HorG,HorB,ZenR,ZenG,ZenB,Expos,Misi,MisDi,MisSta,MisHi,StaR,StaG,StaB,StarDi,StarSi,OfsX,OfsY,OfsZ,SizeX,SizeY,SizeZ,TexR,TexG,TexB,DefVar,Col,Nor,Var (World Ipo) LocX,LocY,LocZ,dLocX,dLocY,dLocZ,RotX,RotY,RotZ,dRotX,dRotY,dRotZ,SizeX,SizeY,SizeZ,dSizeX,dSizeY,dSizeZ,Layer,Time,ColR,ColG,ColB,ColA (Object Ipo) Lens,ClSta,ClEnd (Camera Ipo)

Parameters:
curvename
(type=string)

Returns:
the corresponding IpoCurve, or None.
(type=IpoCurve object)

This function has the functionality to get the actual curve by name as well.

Regardless, I am still getting the quaternion error.

So does the above patch correct a bug in blender?

Thanks,
Frank

#42 Frank,

I am not quite sure what the best solution is for your quaternion problem...
Please email me the .blend and i'll take a look.

#43 Frank,
Notice what's missing in the list of curve names for getCurve. None
of the Quat curves are available. Those contain the bone rotations. I'm
not sure you can classify this as a blender bug, its more of a limitation.
In the code they explicitly only support Object ipo curves for those
operations. The patch for the blender code I posted above adds read-only
support for the Action ipo curves, which includes making QuatX, QuatY,
QuatZ and QuatW available to use in getCurve(), getCurveCurval(), etc.

James,
Just an FYI, if Frank's problem only happens when he's attempting to
export BAKED actions, its might be because the python code isn't finding
any locs or rots at all because of the way has_loc and has_rot are
computed (there must be 3, 4 or 7 ipo curves, baked ipo curves have
10 curves, quat, loc and size). That's one reason I changed the
has_loc and has_rot computations in one of my posts above. My implementation
isn't complete though, one should actually test to see if size or loc
exists if you have 3 curves, I just assumed loc since that's all that was
supported.

As far as size goes ... it does matter, but not a whole lot, :-).
I'd like to see it as I have some action constraints that use this
and I'd just like to see full support. Is there any documentation
on the dsq format ? I can get the size info from blender, but just
wouldn't know how to get it into the dsqs.

#44 I am kind of embarrassed. The error I reported before I was getting before, but because I did not change the path to the script directory it was not running the modified version. With the changes Zaz suggested I am getting errors that deal with ipo.getCurveCurval. It seems that without the changes in blender it returns None when using the text "LocX" or others. I will get the source for blender and try again. On a side note: I did print out the skeleton and could get all the quaternions for each bone after incrementing the frame each iteration. If there is a way to get the names of the bones from the action then it may be desirable to cycle through the action getting the rotations and locations from the bones themselves. It might save a lot of headache.

Thanks for your patience, especially Zaz,
Frank

#45 Zaz,

Firstly, i fixed the Quaternion export bug (just needed to clamp down the values so they didn't overflow)

See Dts_Shape for the .dsq writer. It already writes all the sequence data out into the file.
The Exporter, if dsq's are enabled, loops through all the sequences, getting data for each one and storing it in the Dts_Shape. This data is then saved to the dsq, then deleted(only the stuff currently imported) each time a new sequence is being loaded - The net effect being that you have a dsq for each sequence, containing only that sequences node states and data.

Remember, you don't need DSQ's to make sequences for your shape work in torque. They are merely used as a conveniant way to share animations between shapes with the same skeleton (e.g. player animations).
DTS can happily have sequences stored in the .dts itself.

There appears to be a few alternative arrays for scale data in torque. My guess is that you pick one, make sure that the flags in the sequence reflect the scale type (e.g. UniformScale, AlignedScale...), and of course store the appropriate matters bits for all the nodes affected by scale.

And pseudo example(presuming all your nodes are animated by a sequence) would be :

MySequence = Sequence()
MySequence.flags |= AlignedScale
for n in MyShape.Nodes:
	MySequences.matters_scale.append(0) # Add blank matters

for n in node tracks used in sequence:
	if node animates scale:
		MySequence.matters_scale[node index] = 1 # Animates Scale
		For f in node track frames: # Add scale data
			MyShape.nodeAlignedScales.append(scale vector frame f)
	else:
		MySequence.matters_scale[node index] = 0 # Does not Animate Scale
Shape.sequences.append(MySequence)

Look in the sequence import code in the exporter for a much better example :)

#46 A note about baked animations:
When using print statements to see what my code was failing on a I noticed that my baked animations only have 7 curves. It also only shows 7 curves in the IPO window. I did make the master animation using on LocRot keyframes. The results I am getting is just the position and quaternion values. I am running 2.32 Blender, however.

Thanks,
Frank

#47 James,
Is your fix in a place I can download it from?

Thanks,
Frank

#48 Frank,
No problem, glad you finally noticed the error. Yeah, you'll get
None a lot with those methods as there are some explicit tests
for the curve type inside blender, tests for ID_OB (which is the
Object ipo curves, i.e. animation of the object by moving, rotating,
scaling, etc.). The ipos for Actions have a type of ID_AC.

Did you do your frame iteration test with a BAKED action ? If you
did, I'd expect those results. If you did it without baking the
action, I wouldn't expect those results and my testing didn't get
them. Oh, I'm assuming an armature with constraints involved, i.e.
like IK Solver constraints and likely others. When I tested an
unbaked action on an armature that had IK Solver Constraints involved,
the bones being animated via the IK chain always printed out as being
in their idle/rest position.

It'd be great if it worked that way and I definitely could have futzed
up my test somehow. I only ran it once on the unbaked action and when
I got no rotations or location changes back, I changed my path of work.


James,
Yeah, I'm more comfortable with seeing them there though and haven't
determined how to set the stuff up in the .cs files if I don't have
the file sitting there in front of me, :-). Oh, well, I'm a complete
newb when it comes to torque right now, but as I get my little spurts
of time into it, I pick up more each time.

Thanks for the scale stuff, I was wondering if there wasn't some support
in the .py for them already.

BTW, do you think the importing of DTS shapes into blender should be working
now ? I tried Dts_Import_Blender.py on the demo player.dts and it didn't
work for me. There were several class variables that had been commented
out in one of the other .pys, but referenced in Dts_Import_Blender.py.
Even when I uncommented them, it didn't work for me. I didn't spend much
time on it though, it was just a curiosity and still not hugely important.

#49 Frank,

Download the snapshot again. I have updated it to fix the quaternion bug.

Zaz,

Same goes for you too Zaz. Make sure you have the latest snapshot (testing, fixes, etc)

The importer script should work(on at least 1 mesh in the shape). It just grabs data from the DtsShape + DtsMeshes.
However, its not quite perfect... Not quite had the time to fix it.
Also, some things like the nodes cannot be imported (incomplete armature api).
As for the commented out shape variables... which ones? =/

As for using the dsq's ingame, make a script the same name as your dts (e.g myshape.dts -> myshape.cs), and plonk the following in it

datablock TSShapeConstructor(MyShapeDts)
{
   baseShape = "./myshape.dts";
   sequence0 = "./myshape_foo.dsq name_to_call_foo";
   sequence1 = "./myshape_goo.dsq name_to_call_goo";
};

#50 Zaz,
Sorry if I was not more explicit.

I built a master IK animation sequence on one bone. Then I BAKED it to get another animation sequence. On the BAKED animation sequence I ran the code to print out all the bone quaternions. I was able to see the quaternion values associated to the bone name.

Notice that because it is using the Armature object to see quaternion values in the bones, the bone names themselves do not have the BAKED word in it.

James,
Does that datablock definition for sequences work with StaticShapeData objects? I thought I had tried that, but could not get it to go those objects. I probably did not do it right, I will try again.

BTW, that is how I test my animations. I just create a StaticShapeData object and tell an action to run in its '.cs' file.

Thanks,
Frank

#51 Hey James,
The new version of the exporter works. Just one problem. I can get a baked animation out, but it acts as though the quaternion values are being used as position values. I had setup some rotation on my object, but instead of rotation I get translation. Is there a way to setup a flag so that if it is a BAKED animation it will reorder the reading of the curves? This still suffers from lack of identification of the curve types, but I think we are close for the BAKED routines. I am starting to understand what Zaz is saying now.

I will try rearranging the order of the ipo conversion and see if it works for BAKED commands.

Thanks,
Frank

#52 Sorry about all the posts.

James and Zaz,
I rearranged the following code:
# Modified here

if has_rot:
ipo_rot = Quaternion(
ipo.getCurveCurval(curve_id),
ipo.getCurveCurval(curve_id + 1),
ipo.getCurveCurval(curve_id + 2),
ipo.getCurveCurval(curve_id + 3))
rot = ipo_rot.inverse() * rot
curve_id += 4

if has_loc:
loc += Vector(
ipo.getCurveCurval(curve_id),
ipo.getCurveCurval(curve_id + 1),
ipo.getCurveCurval(curve_id + 2))
curve_id += 3

''' Original
if has_loc:
loc += Vector(
ipo.getCurveCurval(curve_id),
ipo.getCurveCurval(curve_id + 1),
ipo.getCurveCurval(curve_id + 2))
curve_id += 3

if has_rot:
ipo_rot = Quaternion(
ipo.getCurveCurval(curve_id),
ipo.getCurveCurval(curve_id + 1),
ipo.getCurveCurval(curve_id + 2),
ipo.getCurveCurval(curve_id + 3))
rot = ipo_rot.inverse() * rot
curve_id += 4
'''

This allowed the BAKED commands to work. What we need is a flag from the routine that pulls the BAKED word out. We can then use this flag to rearrange the order of the ipo read functions. Not as precise as what Zaz has implemented, but workable entirely from the script.

Thanks,
Frank

#53 James,
I'll grab the snapshot before I play this weekend and then
check to see what if anything I have here should be integrated.
I might play around with a generic way of delving the curve
ordering and chuck that in. I think this can be done with some
choice processing of exceptions and stuff. If that works, then
we don't need a baked vs. unbaked button as the only reason for
that in my thoughts was to change the curve ordering.

If this works, then it might be worth adding a filter for the
bones to ignore (I have one now, but its pretty specific to me)
and possibly a few other things I have in mind.

As for the datablock, I wondered if that worked without the
dsq files. That's what I've been doing, but since they looked
like file names, it wasn't clear what you'd do if you didn't
have .dsq files. Thanks for the tip.

I don't recall off hand what variables, it was the first thing
I tried when I got your code and I haven't revisited it. I'll
try it out with the new snapshot and let you know what I find.
All I was interested in was to import the player.dts and see
how they modeled it to give me a clue as to what they felt was
a good number of polys, the structure they used, etc.

Frank,
Ok, that would make sense then as far as dumping the script
for the baked actions. Just wanted to make sure I didn't mess
up some testing.

That datablock that James pointed out worked for me with
static shapes, but I had to add the dts to the scene as a
static shape and follow a resource on ambients (I was just
trying to play an idle action). Do a searth in resources for
ambient and you'll find one with ambient in the title. It
has a few lines of .cs that define an onAdd() method that
starts the ambient action up when you add the shape as a
static. I'm not sure how you'd do it for another situation,
but assume its similar and planned on crossing that bridge
when I got there, :-).

Hehe, you hacked those if has_XXX/ipo.getCurveCurval() blocks
about the same way I did when I was first working on this. I
only went and hacked up blender after my export still didn't
work after this or even after using the map I set up. But that
was on that complex armature, things work well on a different
armature that's more appropriate for torque. And it works whether
the action is baked or not.

Strange that you're only getting loc and quat curves when you
bake though. I wonder if that has something to do with my
action constraints which introduce some scaling.

#54 Hey I got the BAKED flag to work!
I successfully created a flag that is set when analyzing the name of the IPO. I then was able to attach it to the sequence structure the same way the IPOs are attached to the sequence structure. Then when it gets to the code for using the data out of the curves is does a check on the BAKED flag for that IPO. If it was baked then it reverses the order from "location then quaternion" to "quaternion then location". I had to change things in two files to get it to work. If you want to see what I did then I can mail it to you. I will also attach a "blend" file that shows two animations in it. One is a BAKED action and the other is a regular action. It should be obvious once you look at it.

I definitely agree with Zaz that Blender should be modified to give more options with respect to BAKED actions. However, this hack will get both types in. It may take some tweaking in Blender, but it will work for now.

James if you want I will mail you the modified exporter and the "blend" file.

Thanks everyone,
Frank

#55 Frank,

Personally, i just use the -show tool in torque to test out shapes.
Thanks for the modified code. Every line counts :)

Zaz,

Does the Except code (for bones/animations) not suffice for stopping bones and animations from being exported?

As for the ipo's, i'd rather have some logical way of getting the curve values rather than detecting if its BAKED or not... For example, couldn't the user just have moved the curve order about? :)

On a sidenote, do you have a small screen? Your posts are oddly squashed up in a small area... =/

#56 If by "Except code" you mean that NodeException: block in the .cfg file, no, that's never worked for me from a functional standpoint. As far as it being the method of specifying which nodes and/or actions should be ignored, I'm fine with that. Although I think at some point putting a UI in front of it would be a good idea for the masses, but not necessary for us hacks right now, :-).

I haven't played with the new snapshot yet, but with the 0.5 release, putting bones/nodes into the NodeException: block didn't do anything at all as far as I could see. I didn't look at this very closely as when you mentioned this to me before you thought it would only work for bones/node without animations and that's more or less what I saw.

My solution was to implment this function that is very specific to my normal bone naming conventions:

def shouldIgnore (name):
    if find (name, "IK_") == 0:
        return 1
    elif find (name, "Lock") >= 0:
        return 1
    elif find (name, "Null") >= 0:
        return 1
    elif name [0] == 'C' and find (uppercase, name [1]) >= 0:
        return 1
    elif name == "FacialCtrlBase":
        return 1
    return 0

And then call it in two places:

class BlenderShape(DtsShape):
	scaleFactor = float(1.0) # Scale Factor
	storeDetail = None # Store for out detail levels

	def addBlenderChildren(self, bone, matrix, parentId, indent = 0):
 		# Blender bones are defined in their parent's rotational
 		# space, but relative to the parent's tail.
 		print " " * (indent+2),
 		print "^^ Bone [%s] (parent %d)" % (bone.name,parentId)

		# PREFS:
		# If the Node is banned, stop adding nodes from this branch including this node
		if Prefs.isNodeExcepted(bone.name) == 1 or shouldIgnore (bone.name):
			return

and in BlenderShape.__init__

# Triggers
		# Done via Empty's in blender, being triggered when a keyframe is hit
		# In order to do this, we should look for a Trigger <name> (name is temporary) if we cannot find a bone
		# of that name

		isTrigger = 0

		if Prefs.exportSequences == 1:

			# Loop through all the IPO blocks and build a list of sequences and a list of
			# which nodes/IPO blocks affect each sequence.
			for ipo in Blender.Ipo.Get():
				# Only accept Action IPO blocks
				if ipo.getBlocktype() != 17217:
					continue

				# Ignore IPOs with no data
				if ipo.getNcurves() == 0:
					continue

				# Try to extract the animation name and the bone name from the IPO name.
				name = ipo.getName()
				animation_name, bone_name = Dts_Blender_Prefs.SplitIpoName(name)
				#print "Got", ipo.getNcurves (), " for ipo", name, "->", animation_name, bone_name
				if animation_name == None:
					continue

				if shouldIgnore (bone_name):
					continue
					
				node = self.getNode(bone_name)

				sequence = self.getSequence(animation_name)

With this code in place, all my special bones, i.e. bones without vertices connected to them, don't export. My thoughts for a more general implementation of this were to replace the shouldIgnore implementation with one that reads a list of regular expressions, say like the list stored in the NodeException: section of the .cfg file.

Yeah, I'd prefer some explicit method of gaining access to the proper ipo curves, say like the one I made in the patch. However, I don't think its good to require something like that if it doesn't make it into the bf-blender code base for whatever reasons. Because of changes made in the ipo handling of curves and their names for action ipos in recent releases, at least in 2.32 and maybe a bit earlier, I'm not sure we can do much besides making assumptions. Apparently, from looking at some other python code dealing with this, the string representation of the curves used to include the curves number. Somehow this was lost and you now just get garbage ... without my patches.

#57 Continued post-ran into post size limit

I was thinking we could isolate all of this logic to one function and whatever we implement would at least be isolated. I patterned my first solution off the cal3d exporter, but had to change it because of the change to blender mentioned above.

def BuildCurveMap(ipo):
	curvemap = {}
	ipocurves = ipo.getCurves()
	for i in range(ipo.getNcurves()):
		thiscurveid = ipocurves[i].getName ()
		curvemap[thiscurveid] = i
		#print "ipoCurve:", thiscurveid, " mapping to ", i
	return curvemap

Then in BlenderShape.__init__:

# If we are adding rotation or translation nodes, make sure the
					# sequence matters is properly set..
					sequence.matters_rotation[nodeIndex] = 1
					sequence.matters_translation[nodeIndex] = 1

					#curvemap = BuildCurveMap (ipo)
					# Support Size here (TODO)
					for bez in range(sequence.numKeyFrames):
						time = ipo.getCurveBeztriple(0, bez)[3]
						scene.currentFrame(int(time))

deleted code

						if has_loc:
							loc += Vector(
								ipo.getCurveCurval(curvemap ["LocX"]),
								ipo.getCurveCurval(curvemap ["LocY"]),
								ipo.getCurveCurval(curvemap ["LocZ"]))

						if has_rot:
							ipo_rot = Quaternion(
								ipo.getCurveCurval(curvemap ["QuatX"]),
								ipo.getCurveCurval(curvemap ["QuatY"]),
								ipo.getCurveCurval(curvemap ["QuatZ"]),
								ipo.getCurveCurval(curvemap ["QuatW"]))
							rot = ipo_rot.inverse() * rot

The above implementation doesn't work because of blender's not handling of non Object ipo curves and its from memory so it won't likely work in its current form. However, the idea should be sound in that if you have a function like BuildCurveMap make a mapping of curve names to indexes for the curves, you've at least isolated that code to one easy to find place and can make some efforts in that place to properly determine the correct association between curve names and their order.

However, in an unpatched blender 2.32 and probably a release or two earlier, we have no access to any direct information about which curve is which. In releases previous to that we could use the cal3d exporter trick about looking at the repr() of the curve to get blender's internal number for the curve. (Sheesh, why they eliminated this is not obvious to me.) For current releases though, the only information we have is the number of curves and possibly if the action was baked by checking the name for BAKED or by having the user toggle the Baked atttribute on the sequence panel in the UI.

As far as users reordering the curves ... hmm, lets see, didn't your release doco say that you were only able to use LocRot keys in your actions, :-) ? The only reason for this is/was so you could assume the order of the IpoCurves. Yes, it definitely would be nice to eliminate this, but until blender has some limits removed I don't think we can do much more than be a little smarter and work in a few more specific situations than just the original limiation of only using LocRot keys.

I believe the only influence a user has on IpoCurve order is how they put the keys in. Once you've added, say a Loc key, I don't think there's any way for you to reorder the LocX, LocY and LocZ curves. You could add say a size key, then a rot key and then a loc key to get SizeX/Y/Z, QuatX/Y/Z/W, LocX/Y/Z ordering of the curves. I'm not ever sure if that would happen though, we'd need some testing. I'll play with this and see what I notice since with my patch I have access to the names.

#58 Continued post-ran into post size limit

Sorry about the squashed posts, :-). A web site or two I visit doesn't wrap posts properly under some conditions. For some reason this thread has gotten very wide in my browser and I wasn't letting the text wrap. I am running 1280x1024 with normal size fonts (180 columns wide with XEmacs full width) and this thread is now almost as wide as my entire screen. I'll make my browser window wide enough to see the entire width of the thread and fight my urge to hit the Enter key at the end of the line, :-).

#59 Okie,

Firstly, i already have the CurveMap code in the exporter; However, i don't use it, since as you have quite much put it, i can only get Loc* Curve names with it.

I suppose if worst comes to worst, we could have a little torque specific fork of blender... hehe :)

#60 Yeah, when I looked at the cal3d exporter I recognized the curve mapping you had in there and tried to make the cal3d stuff work. That's when I looked at blender and saw they changed the string representation code, or more accurately what it used, which broke the cal3d exporter curve mapping also. Of course, if we "fork" blender, one way or another, this is basically a mute point, :-).

Actually, I was initially against forking or requiring patches as I was thinking about it from a "how would the masses" feel about it. The truth is that everyone who would use this should either be a programmer or have one handy if they're going to use this. Requiring someone to run a few small patches against the bf-blender code base and build it wouldn't be much of an issue. Or, since the blender executables are small, they could maybe even just be available for download if you have web space for another 5M or so. I'd be happy to provide linux versions of the executable.

If we were to keep the fork as a small set of patches applied against the original bf-blender code base and try to get our patches applied to it, this probably isn't that bad of a way to go. Especially when it comes to something like these curve names. Its pretty obvious that if it isn't actually a bug, its at least a limitation that should be removed from bf-blender. Sometime in the future this should problem should be removed from bf-blender and at that point our "fork" might just vanish.

One thing I definitely don't want to do is to go the instinctive route. Intrr has done some very good work, but its too bad his version is missing out on so many of the new things in blender, but then he has different goals for doing what he's done than we do and his changes are much greater than I'd ever expect us to have to make for this. Hmm, maybe we need to integrate torque as a replacement for the game engine in blender ??? Naw, maybe not, :-).

If you want to go this way, I'd gladly email you a proper patch that you could include with the release. The question would be which version of blender we'd diff from. Would we follow the current CVS or would we have the patch(es) patch against the latest release ?

I general run the current CVS version I build myself with optimizations for my box, so it'd be easiest for me if we were to go against the current CVS version. But then, patching against a release might be safer, especially if we somehow made downloads of the executables available.