Advice on Organizing Lots of Data?
by Aaron Miller · in Torque Game Builder · 01/16/2009 (9:41 pm) · 7 replies
I'm trying to save myself a bit of future pain before I stumble down a blind path.
Can anyone recommend the best way to handle large amounts of text and numerical data in TorqueScript? I am trying to plan out how I will deal with game items which have a number of properties (such as trade goods or weapons which have 8 to 12 stats or characteristics each).
Right now I'm using SimObjects and growing more comfortable with them. They seem like a very flexible method which will allow me to add and remove characteristics as the game develops. But I imagine a very large number of objects (speculating on two or three hundred, but that may be wildly unrealistic) and I'm wondering if I need to get serious about looking into how to integrate a database into TGB.
Thanks for any advice!
Can anyone recommend the best way to handle large amounts of text and numerical data in TorqueScript? I am trying to plan out how I will deal with game items which have a number of properties (such as trade goods or weapons which have 8 to 12 stats or characteristics each).
Right now I'm using SimObjects and growing more comfortable with them. They seem like a very flexible method which will allow me to add and remove characteristics as the game develops. But I imagine a very large number of objects (speculating on two or three hundred, but that may be wildly unrealistic) and I'm wondering if I need to get serious about looking into how to integrate a database into TGB.
Thanks for any advice!
#2
IE
The player walks into a dungeon, have some way to tell the game what objects/creature templates will be needed, then load outside text files into sim objects.
You like?
01/18/2009 (10:13 am)
Text files. File i/o to retrieve info. You can use your simobjects to organize the item descriptions that you will need for any given level.IE
The player walks into a dungeon, have some way to tell the game what objects/creature templates will be needed, then load outside text files into sim objects.
You like?
#3
(I'll have to keep in mind that old programmer quote, "premature optimization is the root of all evil.")
01/20/2009 (12:27 pm)
Thanks Kevin. I don't know why I was overlooking text, just thinking it couldn't be THAT easy. :) I've been worried about data organization and making changes, but there's no reason why I couldn't maintain a database that spits out delimited files.(I'll have to keep in mind that old programmer quote, "premature optimization is the root of all evil.")
#4
eg, from the console:
and here is masterlist.cs:
01/20/2009 (1:31 pm)
note for serialization to/from disk, you also have SimObject.save() and exec().eg, from the console:
==>new simGroup(masterList);
==>masterList.myProperty = 100;
==>masterList.add(new SimGroup() { myProperty = "heck yeah"; });
==>masterList.add(new SimGroup() { myProperty = "you betcha"; });
==>masterlist.save("projects/masterlist.cs");
==>quit();(restart)exec("projects/masterlist.cs");
==>masterlist.dumpTree();
9687-SimGroup-masterList
9688-SimGroup---
9689-SimGroup---and here is masterlist.cs:
//--- OBJECT WRITE BEGIN ---
new SimGroup(masterList) {
myProperty = "100";
new SimGroup() {
myProperty = "heck yeah";
};
new SimGroup() {
myProperty = "you betcha";
};
};
//--- OBJECT WRITE END ---(dumpTree() is from this resource.)
#5
i would not balk at two or three hundred objects,
as far as script data structures go. it will handle that fine.
vSide has a datastructure which is constructed in script with about 3200 entries, each with maybe eight data fields.
you do want to avoid iterating over a large number of items often, tho.
ie, if you're parsing 200 or 300 things at startup or at mission connect or something, that's fine, but you probably wouldn't want to do it say as a result of player action. if you do find yourself iterating over all those entries fairly often, you might want to rethink the data design and/or consider doing the iterating in C.
01/20/2009 (1:34 pm)
also fwiw,i would not balk at two or three hundred objects,
as far as script data structures go. it will handle that fine.
vSide has a datastructure which is constructed in script with about 3200 entries, each with maybe eight data fields.
you do want to avoid iterating over a large number of items often, tho.
ie, if you're parsing 200 or 300 things at startup or at mission connect or something, that's fine, but you probably wouldn't want to do it say as a result of player action. if you do find yourself iterating over all those entries fairly often, you might want to rethink the data design and/or consider doing the iterating in C.
#6
Have you thought about XML? It may be quite useful as it may be edited in Excel or OpenOffice Calc easily. You can find documentation and definitions for XML class in ./common/gameScripts/xml.cs and also an example somewhere inside "common" folder.
Also keeping everything in SimObject with good accessors is fine idea.
01/20/2009 (2:55 pm)
Hi!Have you thought about XML? It may be quite useful as it may be edited in Excel or OpenOffice Calc easily. You can find documentation and definitions for XML class in ./common/gameScripts/xml.cs and also an example somewhere inside "common" folder.
Also keeping everything in SimObject with good accessors is fine idea.
#7
Seriously.. I'm running something around 4000 simobjects and they work great! When I save it to disk it makes a 25,000 line text file in short order too. You can save them to disk as a text file and load them back up! Just don't go thinking that you can pass them through TGB's sad network layer at a blazing speed and you'll be a-ok for a few thousand more objects.
Keep it simple and maybe you'll finish a project.. at least that's my motto lately.
Toodles!
-nic
01/26/2009 (4:48 pm)
Dood... my humble opinion is to keep what you're doing. Seriously.. I'm running something around 4000 simobjects and they work great! When I save it to disk it makes a 25,000 line text file in short order too. You can save them to disk as a text file and load them back up! Just don't go thinking that you can pass them through TGB's sad network layer at a blazing speed and you'll be a-ok for a few thousand more objects.
Keep it simple and maybe you'll finish a project.. at least that's my motto lately.
Toodles!
-nic
Torque 3D Owner Aaron Miller