Taml, dear Taml
by Lukas Joergensen · 07/06/2014 (2:51 pm) · 20 comments
Can you guess what makes the above image special?.. Nothing actually, it's what happens under behind the scenes thats interesting, and what makes the above image interesting is the fact that theres nothing special about it!
In TerrainEditToolbar.gui I removed the whole "EWTerrainEditToolbar" object and replaced it with a single line:
TamlRead("test.taml");Well, what is this Taml stuff? It's something I stole from T2D.. How does it work?
I wrote the following command in the console:
TamlWrite(EWTerrainEditToolbar, "test.taml");
Which allowed me to export the whole GUI object into a neat little xml file. The remarkable thing is, it didn't change a thing. It's working exactly as it did before the only change is that the GUI is written in XML now instead of TorqueScript!
A sample of the generated XML file:
<GuiControl
Name="EWTerrainEditToolbar"
position="306 0"
extent="800 40"
minExtent="8 2"
horizSizing="right"
vertSizing="bottom"
hovertime="1000"
isContainer="1"
internalName="TerrainEditToolbar"
canSaveDynamicFields="0">
<GuiTextCtrl
text="Brush Settings"
maxLength="255"
margin="0 0 0 0"
padding="0 0 0 0"
anchorTop="1"
anchorBottom="0"
anchorLeft="1"
anchorRight="0"
position="6 7"
extent="70 16"
minExtent="8 8"
horizSizing="right"
vertSizing="bottom"
hovertime="1000"
isContainer="1"
canSaveDynamicFields="0" />
// ...
</GuiControl>You can see the whole file here You can compare that to the original TorqueScript version here.
If you wonder what Taml actually is, you should read up on some of the T2D blogs, as it is a T2D thing. I can say that it is a system to serialize SimObjects to different formats (e.g. XML, Binary or JSON).
The TAML implementation can be found at my branch on GitHub feel free to pick it up, but don't expect it to be very a "nice" or "clean" implementation. It was made as a sideproject and it was a messy implementation as I frankly didn't really know what I was doing so it is a bit dirty here and there.
About the author
IPS Bundle available at: https://www.winterleafentertainment.com/Products/IPS.aspx
#3
I have only tested it with the EWTerrainEditToolbar and SceneObjects. I had to make some small changes to make SceneObjects export position, rotation and scale properly so I'm sure that there are other fields that doesn't get exported properly, but I don't really have an overview of it atm.
I think it'll be a trial and error process of simply exporting objects and see what is missing :P
07/06/2014 (3:14 pm)
The formatting is quickly fixed with a "find and replace" ;)I have only tested it with the EWTerrainEditToolbar and SceneObjects. I had to make some small changes to make SceneObjects export position, rotation and scale properly so I'm sure that there are other fields that doesn't get exported properly, but I don't really have an overview of it atm.
I think it'll be a trial and error process of simply exporting objects and see what is missing :P
#4
Since the following if-statement will always result in "continue".
It's one of my issues currently, as the only solution I have found is change it so that it returns true when necessary.
07/06/2014 (4:30 pm)
In the hope of finding someone familiar with the Taml system, since "setPosition" always returns false how does the position ever get written to a .taml file?Since the following if-statement will always result in "continue".
if ( elementCount == 1 &&
pField->writeDataFn != NULL &&
( !getWriteDefaults() && pField->writeDataFn( pSimObject, fieldName ) == false) )
continue;It's one of my issues currently, as the only solution I have found is change it so that it returns true when necessary.
#5
In T2D, all static fields have a write function defined for them. This can be seen in the initPersistFields method of each SimObject derived class (here's the position field in SceneObject.cc):
Then in each classes' header file you can find the write function with the condition defined which specifies when Taml writes that field out:
So in this case, any position that is not "0 0" is written out.
07/07/2014 (12:03 am)
Taml is really great, nice work porting this.In T2D, all static fields have a write function defined for them. This can be seen in the initPersistFields method of each SimObject derived class (here's the position field in SceneObject.cc):
addProtectedField("Position", TypeVector2, NULL, &setPosition, &getPosition, &writePosition, "");Then in each classes' header file you can find the write function with the condition defined which specifies when Taml writes that field out:
static bool writePosition( void* obj, StringTableEntry pFieldName ) { return static_cast<SceneObject*>(obj)->getPosition().notZero(); }So in this case, any position that is not "0 0" is written out.
#6
github.com/GarageGames/Torque2D/issues/79
github.com/GarageGames/Torque2D/issues/94
Just stuff to be aware of during T3D Taml testing, and if you fix any of these issues, send a pull request our way too. :)
07/07/2014 (12:19 am)
For completeness, there's also a few open issues regarding Taml that we are tracking on Github. github.com/GarageGames/Torque2D/issues/79
github.com/GarageGames/Torque2D/issues/94
Just stuff to be aware of during T3D Taml testing, and if you fix any of these issues, send a pull request our way too. :)
#7
Thanks for the heads-up on those two, I'll tell you if I run into/fix those issues!
07/07/2014 (12:27 am)
Ugh, didn't see it said "writeDataFn", I guess I must just have changed it to set while I was on auto-pilot mode, changing it to make it compile :PThanks for the heads-up on those two, I'll tell you if I run into/fix those issues!
#8
07/07/2014 (6:56 am)
I remember there being issues with missing fields in the output, and there are occasionally still bugs with incorrectly initialized fields on read in T2D. As Mike points out. Lots of little things to track down! Thanks for the effort!
#9
Now I've got default values working properly, which cropped the taml output down from 420 lines, to 267! (Original TorqueScript version was 467)
You can find the new and cleaner taml output here!
I've also implemented CustomFields for Rivers, MeshRoads, WaterObjects, CloudLayers and ConvexShapes:
So much cleaner than:
At least imo.
07/11/2014 (4:20 am)
Just added ~20 commits to my repo.Now I've got default values working properly, which cropped the taml output down from 420 lines, to 267! (Original TorqueScript version was 467)
You can find the new and cleaner taml output here!
I've also implemented CustomFields for Rivers, MeshRoads, WaterObjects, CloudLayers and ConvexShapes:
<CloudLayer
Name="SampleCloudLayer"
Texture="art/skies/clouds/clouds_normal_displacement"
TexScale="1"
texDirection="0.5 0"
texSpeed="0.005"
baseColor="0.9 0.9 0.9 1"
exposure="1"
coverage="0.2"
windSpeed="1"
height="4"
Position="78 -306 2">
<CloudLayer.Textures>
<Texture
Scale="0.05"
Direction="1 0"
Speed="0.005" />
<Texture
Scale="0.5"
Direction="0 1"
Speed="0.005" />
<Texture
Scale="1"
Direction="0.5 0"
Speed="0.005" />
</CloudLayer.Textures>
</CloudLayer>So much cleaner than:
new CloudLayer(SampleCloudLayer) {
Texture = "art/skies/clouds/clouds_normal_displacement";
TexScale[0] = "0.05";
TexScale[1] = "0.5";
TexScale[2] = "1";
texDirection[0] = "1 0";
texDirection[1] = "0 1";
texDirection[2] = "0.5 0";
texSpeed[0] = "0.005";
texSpeed[1] = "0.005";
texSpeed[2] = "0.005";
baseColor = "0.9 0.9 0.9 1";
exposure = "1";
coverage = "0.2";
windSpeed = "1";
height = "4";
Position = "78 -306 2";
Rotation = "1 0 0 0";
Scale = "1 1 1";
canSave = "1";
canSaveDynamicFields = "1";
};At least imo.
#10
07/11/2014 (10:00 am)
I see that we're still casting booleans to "1" and "0", and writing numeric values as strings :P. The TorqueScript is strong in this one...
#11
07/11/2014 (12:51 pm)
Hah i can see your point, i can fix the boolean stuff but i dont know if we can avoid writing numerics as string, depends on TinyXML, and how it parses the attributes.
#12
But yes, it is much cleaner than having those arrays in there manually :). A bit more verbose, but hey, it's XML :P.
07/12/2014 (3:00 am)
Oh, is it not the responsibility of the console to tell TinyXML what to write? I figured it was just a lack of console type information somewhere in the pipe.But yes, it is much cleaner than having those arrays in there manually :). A bit more verbose, but hey, it's XML :P.
#13
Which is also why this probably wont make it into stock T3D :P I got so tried of spending time trying to work around modifying the TinyXML library that i just ended up modifying it, as i don't think OMNI will have similar restrictions :P
07/12/2014 (3:07 am)
I believe TinyXML always uses the "-marks around the attribute values such that it can clearly see when the value starts and stops. We could modify the TinyXML library to handle this differently, but knowing how the t3d community feels about modifying libraries, i think the modifications i have already made is enough.Which is also why this probably wont make it into stock T3D :P I got so tried of spending time trying to work around modifying the TinyXML library that i just ended up modifying it, as i don't think OMNI will have similar restrictions :P
#14
We will just want it tested and proven.... but you know this.
07/12/2014 (7:00 am)
You know we wont :PWe will just want it tested and proven.... but you know this.
#15
Just out of curiousity, did JSON reading ever really work in T2D? Maybe Taml was developed in an IDE which auto-escapes buffers, but VS does no such thing :P
07/12/2014 (8:50 am)
@Lilligreen Sent some PR's in T2D's general direction. Just out of curiousity, did JSON reading ever really work in T2D? Maybe Taml was developed in an IDE which auto-escapes buffers, but VS does no such thing :P
#16
07/12/2014 (9:43 am)
@Lukas : never tried it specifically but Json format is in as of T2D 3.0 and is used by the Spine Animation implementation to read in texture atlases (.atlas) and skeleton files (.json) which are both JSON format.
#17
Although it does seem like there was an intention of escaping it properly, because it seems that it allocates room for an extra byte, it just doesn't add a \0 to that extra byte. (Which is what I tried to fix here)
07/12/2014 (9:50 am)
@Simon hmm weird, when I run it on Windows it complains about the JSON files not being escaped properly, perhaps the ones it import from Spine has the \0 char in the file, but JSON files exported by T2D doesn't have the \0 in the end, and there is nothing that ensures that it is escaped properly.Although it does seem like there was an intention of escaping it properly, because it seems that it allocates room for an extra byte, it just doesn't add a \0 to that extra byte. (Which is what I tried to fix here)
#18
I implemented a custom version of the writeDataFn thing, so it's easier to write "default value" methods for fields.
It uses a struct, so you have to create an instance of the struct for each of the fields, but I don't think that will add much of a memory overhead.
Example:
There are also more advanced structs available to bind the testing directly to a method or field:
Edit: Biggest problem with this is that it deviates from how T2D does things.. Which is how btw? Do you write a custom function for each of them?
07/13/2014 (1:30 pm)
Okay small update:I implemented a custom version of the writeDataFn thing, so it's easier to write "default value" methods for fields.
It uses a struct, so you have to create an instance of the struct for each of the fields, but I don't think that will add much of a memory overhead.
Example:
addField( "decalType", TypeTSMeshType, Offset( mDecalType, TSStatic ),
// Compare the value as a string to the default value given.
new DefaultValueWriteFn("Collision Mesh"),
"The type of mesh data used to clip decal polygons against." );
addField( "allowPlayerStep", TypeBool, Offset( mAllowPlayerStep, TSStatic ),
// Compare the value as a bool to the default value given.
new DefaultBoolWriteFn(true),
"@brief Allow a Player to walk up sloping polygons in the TSStatic (based on the collisionType).nn"
"When set to false, the slightest bump will stop the player from walking on top of the object.n");There are also more advanced structs available to bind the testing directly to a method or field:
addProtectedField( "position", TypeMatrixPosition, Offset( mObjToWorld, SceneObject ), &_setFieldPosition, &defaultProtectedGetFn, // Get the value using the given method, and compare it // to the given default value. new PublicConstMethodWriteFn<SceneObject, Point3F>(Point3F::Zero, &SceneObject::getPosition), "Object world position." );
Edit: Biggest problem with this is that it deviates from how T2D does things.. Which is how btw? Do you write a custom function for each of them?
#19
In T2D, all static fields have their own custom write function, at least in theory. The Gui system is the biggest hole we have at the moment with missing write functions. Without a custom function, Taml writes out that field anyway. The goal of the function is to determine if the field's value is at its default, and if it is, it is skipped. That way the files are not so verbose and can be read back in quicker.
So for example in SimObject:
A custom write function is defined and included in the header file:
07/14/2014 (9:00 am)
Thanks for the pull requests Lukas, I'll look at them in detail soon.In T2D, all static fields have their own custom write function, at least in theory. The Gui system is the biggest hole we have at the moment with missing write functions. Without a custom function, Taml writes out that field anyway. The goal of the function is to determine if the field's value is at its default, and if it is, it is skipped. That way the files are not so verbose and can be read back in quicker.
So for example in SimObject:
addField("canSaveDynamicFields", TypeBool, Offset(mCanSaveFieldDictionary, SimObject), &writeCanSaveDynamicFields, "");A custom write function is defined and included in the header file:
static bool writeCanSaveDynamicFields( void* obj, StringTableEntry pFieldName ) { return static_cast<SimObject*>(obj)->mCanSaveFieldDictionary == false; }
#20
This would be the equivalent for that writeFunction you just wrote:
It's slightly slower but if you want to do it exactly the same way you could do this:
Which imo is just easier to manage as you can do it all in-line.
07/14/2014 (9:11 am)
Yeah thats what I wanted to avoid because it felt pretty tedious to write those functions over and over.This would be the equivalent for that writeFunction you just wrote:
new DefaultBoolWriteFn(false);
It's slightly slower but if you want to do it exactly the same way you could do this:
new PublicMemberWriteFn<SimObject, bool>(false, &SimObject::mCanSaveFieldDictionary);
Which imo is just easier to manage as you can do it all in-line.
Employee Michael Perry
ZombieShortbus
Is everything hooked up? GUIs, objects, missions, materials, etc?