XML Persistance for Torque Objects
by J. Donavan Stanley · 05/19/2003 (11:03 am) · 35 comments
Download Code File
Installation
1) Unzip the archive into your engine directory.
2) Add the files in engine/persistance to your project.
3) Recompile.
Info
This introduces two new classes to the Torque Engine:
Pickler - An abstract base class for serialization support.
XMLPickler - A concrete class that can serialize objects to/from XML.
This approach was taken so that others could implement custom picklers using XMLPickler as a guide. For example someone may want to implement a BinaryPickler that stores objects in a non-human-readable format.
If you unpickle an object that already exists within the simulation it's fields will be overwritten with the contents of the XML stream.
Only fields that are exposed to the scripting engine (via addField ) are imported / exported.
You must specify a storage medium as part of the desitination. For example "file://test.xml" refers to the file "test.xml" in the root directory of the Torque executable. Currently only "file" storage mediums are supported.
You can see a sample output file by clicking here.
This uses a modified version of the TinyXML library.
Usage
In a nutshell:
1) Create a XMLPickler object.
2) Call either pickle (passing an object and a desitination), or unpickle(passing in a source).
3) You're done.
Script Example:
Installation
1) Unzip the archive into your engine directory.
2) Add the files in engine/persistance to your project.
3) Recompile.
Info
This introduces two new classes to the Torque Engine:
Pickler - An abstract base class for serialization support.
XMLPickler - A concrete class that can serialize objects to/from XML.
This approach was taken so that others could implement custom picklers using XMLPickler as a guide. For example someone may want to implement a BinaryPickler that stores objects in a non-human-readable format.
If you unpickle an object that already exists within the simulation it's fields will be overwritten with the contents of the XML stream.
Only fields that are exposed to the scripting engine (via addField ) are imported / exported.
You must specify a storage medium as part of the desitination. For example "file://test.xml" refers to the file "test.xml" in the root directory of the Torque executable. Currently only "file" storage mediums are supported.
You can see a sample output file by clicking here.
This uses a modified version of the TinyXML library.
Usage
In a nutshell:
1) Create a XMLPickler object.
2) Call either pickle (passing an object and a desitination), or unpickle(passing in a source).
3) You're done.
Script Example:
new XMLPickler( "pickler" ); pickler.pickle( LightMaleHumanArmor, "file://test.xml" ); $obj = pickler.unpickle( "file://test.xml" );
About the author
#22
This version is much better.
06/20/2003 (7:49 pm)
I'm not quite what I was smoking when I wrote that. It was a comepletly boneheaded implementation.This version is much better.
bool TiXmlDocument::LoadStream( FileStream& stream )
{
// Delete the existing data:
Clear();
U32 length = stream.getStreamSize();
char* data = new char[length + 1];
stream.read( length, data);
data[length] = 0;
Parse( data );
delete data;
if( Error() )
return false;
else
return true;
}
#23
...
delete data;
...
in the above function... any idea why?
Here's the callstack...
07/30/2003 (12:00 pm)
hm, DEBUG build crashes on me at...
delete data;
...
in the above function... any idea why?
Here's the callstack...
NTDLL! 77897704()
PlatformAssert::process(PlatformAssert::Type Fatal, const char * 0x00b4cd80 `string', unsigned int 910, const char * 0x00c11eec buffer) line 86
PlatformAssert::processAssert(PlatformAssert::Type Fatal, const char * 0x00b4cd80 `string', unsigned int 910, const char * 0x00c11eec buffer) line 105
Memory::free(void * 0x0469c4b0, unsigned char 0) line 910 + 83 bytes
operator delete(void * 0x0469c4b0) line 1105 + 11 bytes
TiXmlDocument::LoadStream(FileStream & {...}) line 773 + 15 bytes
TiXmlDocument::LoadFile(const char * 0x013242af) line 830
XMLPickler::unpickleFromFile(const char * 0x013242af, SimObject * 0x00000000) line 95 + 12 bytes
Pickler::unpickle(const char * 0x013242a8, SimObject * 0x00000000) line 70 + 36 bytes
cPicklerunpickle(SimObject * 0x023b8aa0, int 3, const char * * 0x00b96378) line 296 + 24 bytes
CodeBlock::exec(unsigned int 1895, const char * 0x01f8defc, Namespace * 0x0191a050, unsigned int 3, const char * * 0x00b96378, unsigned char 0) line 1151 + 23 bytes
CodeBlock::exec(unsigned int 1828, const char * 0x017c26b4, Namespace * 0x0191a050, unsigned int 2, const char * * 0x00b96378, unsigned char 0) line 1078
CodeBlock::exec(unsigned int 3682, const char * 0x017c1ddc, Namespace * 0x01e7ede0, unsigned int 2, const char * * 0x00b96378, unsigned char 0) line 1078
CodeBlock::exec(unsigned int 6393, const char * 0x00bb72e4 char * RemoteCommandEvent::mBuf, Namespace * 0x016e7028, unsigned int 2, const char * * 0x023b85cc, unsigned char 0) line 1078
Namespace::Entry::execute(int 3, const char * * 0x023b85cc, ExprEvalState * 0x00b9e5c0 class ExprEvalState gEvalState) line 779 + 40 bytes
Con::execute(int 3, const char * * 0x023b85cc) line 746
RemoteCommandEvent::process(NetConnection * 0x041f47c0) line 147 + 22 bytes
NetConnection::eventReadPacket(BitStream * 0x00ba7ff8 gPacketStream) line 308 + 17 bytes
NetConnection::readPacket(BitStream * 0x00ba7ff8 gPacketStream) line 645
GameConnection::readPacket(BitStream * 0x00ba7ff8 gPacketStream) line 848
NetConnection::handlePacket(BitStream * 0x00ba7ff8 gPacketStream) line 514 + 17 bytes
ConnectionProtocol::processRawPacket(BitStream * 0x00ba7ff8 gPacketStream) line 226 + 17 bytes
NetConnection::processRawPacket(BitStream * 0x00ba7ff8 gPacketStream) line 487
NetConnection::sendPacket(BitStream * 0x00ba7ff8 gPacketStream) line 627 + 34 bytes
NetConnection::checkPacketSend(unsigned char 0) line 611 + 17 bytes
NetInterface::processClient() line 400
DemoGame::processTimeEvent(TimeEvent * 0x0012fe60) line 684
GameInterface::processEvent(Event * 0x0012fe60) line 72 + 17 bytes
GameInterface::postEvent(Event & {...}) line 153 + 17 bytes
TimeManager::process() line 1222 + 23 bytes
DemoGame::main(int 1, const char * * 0x01311230) line 493
run(int 1, const char * * 0x01311230) line 1139 + 26 bytes
main(int 1, const char * * 0x01311230) line 1175 + 13 bytes
mainCRTStartup() line 206 + 25 bytes
KERNEL32! 77e8ca90()
#24
08/10/2003 (8:24 pm)
No clue Stefen. I've not had it crash yet. If you want to send me the XML stream i'll see if I can repro it.
#25
03/11/2004 (10:26 am)
Any update on this? Did you ever extend it to save dynamic/tagged fields, too? I just had a quick look at dump() in SimBase, but I dont really get it how the tagged fields could be accessed from the Pickler... gotta investigate some more....
#26
07/18/2004 (2:58 pm)
Any new info on this?
#27
Error 1 error C2039: 'getTypeName' : is not a member of 'Con' c:\Torque\engine\persistance\pickle.cc 251
Any ideas?
Nick
01/02/2006 (3:51 am)
I'd like to give this a try but I think Con::getTypeName() isn't used anymore in TGE 1.4. I get this error when compiling:Error 1 error C2039: 'getTypeName' : is not a member of 'Con' c:\Torque\engine\persistance\pickle.cc 251
Any ideas?
Nick
#28
Thanks for help,
Norv
01/10/2006 (11:01 am)
JD, Wendell & Dave - in regard to "TiXmlDocument::LoadStream()" in your discussion of this modification you three know where this is located but you don't indicate the location. Is it in "pickle.cc"? I can't seem to find it. Is this an added function or modified and name changed?Thanks for help,
Norv
#29
03/14/2006 (12:26 pm)
I'm writing code that would allow the use of XML to define GUIs and then convert those XML files into .gui's for use in a TGE game. Would this resource help in that endeavor, or would is this going in a completely different direction? 
#30
:-)
03/15/2006 (1:11 am)
I'd like to see that resource. It would be very useful for me @Jonathon.:-)
#31
http://www.garagegames.com/index.php?sec=mg&mod=resource&page=view&qid=9031
03/15/2006 (5:24 pm)
For those of you following along at home, there is a related thread here:http://www.garagegames.com/index.php?sec=mg&mod=resource&page=view&qid=9031
#32
delete[] data;
You have to use delete[] in c++ to dealloc arrays.
05/30/2006 (9:47 pm)
It is crashing because the line should be :delete[] data;
You have to use delete[] in c++ to dealloc arrays.
#33
10/11/2006 (11:25 am)
I'd like to use this with a T2D/Torque Game Builder project - how should I go about adding this to the source of TGB?
#34
Does unpickle adds dynamic fields to the object from the xml.
thank you.
03/05/2007 (12:34 am)
Unpickle function doesn't work. In console(debug) i get the message can not instantiate non-conobject. Pickle works fine. Does anyone has any idea. I use TGB 1.1.3.Does unpickle adds dynamic fields to the object from the xml.
thank you.
#35
Reading the data (unpickle) works with TGEA 1.0.3 and the following change in pickle.cc line 251:
Change
to
11/16/2007 (2:28 pm)
Thanks for this resource!Reading the data (unpickle) works with TGEA 1.0.3 and the following change in pickle.cc line 251:
Change
expandEscape(textbuf, dstr);
[b] field.SetValue( Con::getTypeName(itr->type) );[/b]
field.SetAttribute( NAME_FIELD, itr->pFieldname );
field.SetAttribute( TYPE_FIELD, (int)itr->type );to
expandEscape(textbuf, dstr);
[b]field.SetValue( FIELD_NODE );[/b]
field.SetAttribute( NAME_FIELD, itr->pFieldname );
field.SetAttribute( TYPE_FIELD, (int)itr->type ); 
Torque Owner Dave Bacher
// we know how long the data is -- no need to read
// in small pieces.
char * buf = new char[ length + 1 ]; // allocate the right number of bytes.
stream.read(length, buf);
buf[length] = 0;
data = buf;
delete buf;
This ought to even run faster, since it doesn't have to make 800000 function calls to read a big file.
Edit: And yes, this means that there are briefly two complete copies of the XML document; in exchange, you're losing more than 1600 function calls, including (potentially) calls to strcat in operator +=, an extremely slow operation as strings become longer. Even if += is implemented to not use strcat, you're still going to cut off a lot of branches and function calls, so from a speed standpoint, the load will take less time.