Lots of little datablocks?
by Daniel Buckmaster · in General Discussion · 04/12/2009 (1:41 pm) · 7 replies
I've implemented some stances into my heavily altered Player class. Some being 32, precisely. I've only made use of about 16 stances so far, but I want to make sure I have room.
Anyway, the thing is, I need to store a lot of data per stance - movement forces and maximum speeds in three directions, jump force and a bunsh of jump data, box size, surface data, etc. This makes for a lot of data in the datablock. It also means when I try to write all that data, I overflow the BitStream very quickly, even if I'm writing a bunch of "" strings.
I've decided that a possible way to advance is to get my datablock to multiply - make lots of litle datablocks, each holding just a bit of the information. For one, this means I can reuse all th information in a more modular fashion, but it also beings the Player construction more in line with, for example, the Explosion system - your average Explosion datablock refers to particle emitter datablocks, which store particle datablocks. It's not all stored in one big datablock.
Is this a healthy way to proceed? Are there downsides to having datablocks that breed like rabbits?
Anyway, the thing is, I need to store a lot of data per stance - movement forces and maximum speeds in three directions, jump force and a bunsh of jump data, box size, surface data, etc. This makes for a lot of data in the datablock. It also means when I try to write all that data, I overflow the BitStream very quickly, even if I'm writing a bunch of "" strings.
I've decided that a possible way to advance is to get my datablock to multiply - make lots of litle datablocks, each holding just a bit of the information. For one, this means I can reuse all th information in a more modular fashion, but it also beings the Player construction more in line with, for example, the Explosion system - your average Explosion datablock refers to particle emitter datablocks, which store particle datablocks. It's not all stored in one big datablock.
Is this a healthy way to proceed? Are there downsides to having datablocks that breed like rabbits?
About the author
Studying mechatronic engineering and computer science at the University of Sydney. Game development is probably my most time-consuming hobby!
#2
what's the mechanism for this ?
too many bits in a single packUpdate() ?
04/13/2009 (8:04 am)
> I overflow the BitStream very quicklywhat's the mechanism for this ?
too many bits in a single packUpdate() ?
#3
I originally used strings so I could pack multiple values into one datablock value where I'm dealing with 2D arrays in the datablock itself. For example, I need an array of maximum speeds per stance per movement mode (walk/run/sprint). I would set up the datablock with a string containing three speeds, the data for each stance's maximum speeds, then break this into three float values after scripts had been loaded.
By making a datablock for the physical data per stance, I avoid the 2D array. This also does introduce some reusability - I can imagine that many Player datablocks will utilise each of these smaller physics datablocks. So there's reduced redundancy there.
Orion - yes, I would run into an 'out of range write' error. I tried setting the maximum length for my strings to 32 instead of the default 255, but that doesn't seem to actually save space when writing strings shorter than that length :P. Which probably bodes well...
Anyway, I thought about using a ResizeBitStream or even an InfiniteBitStream, but I realised that I have no idea how I would go about doing so, or whether that was an appropriate response :P.
04/13/2009 (9:30 am)
Stefan - thanks for the heads-up. I realise that my Explosion example was probably a bad point of reference. I just latched onto that as a precedent for having 'lots of little datablocks' ;P.I originally used strings so I could pack multiple values into one datablock value where I'm dealing with 2D arrays in the datablock itself. For example, I need an array of maximum speeds per stance per movement mode (walk/run/sprint). I would set up the datablock with a string containing three speeds, the data for each stance's maximum speeds, then break this into three float values after scripts had been loaded.
By making a datablock for the physical data per stance, I avoid the 2D array. This also does introduce some reusability - I can imagine that many Player datablocks will utilise each of these smaller physics datablocks. So there's reduced redundancy there.
Orion - yes, I would run into an 'out of range write' error. I tried setting the maximum length for my strings to 32 instead of the default 255, but that doesn't seem to actually save space when writing strings shorter than that length :P. Which probably bodes well...
Anyway, I thought about using a ResizeBitStream or even an InfiniteBitStream, but I realised that I have no idea how I would go about doing so, or whether that was an appropriate response :P.
#4
12.04 = 1-2 bytes as float, 5 bytes as string.
459023 = 3 bytes as integer, 6 bytes as string.
And so forth. ResizeBitStream or InfiniteBitStream won't help you here.
What I would do if I were you, unless you plan to support the download-as-you-go modding, is to get rid of the datablock pack/unpack phase completely. That way you don't have to worry at all and your game will load faster.
We've done this in all our prototypes so far and it's not too complicated apart from a few ParticleEmitter changes in preload ().
04/13/2009 (11:32 am)
Huh? You're actually wasting lots and lots of space when you pack these values into a string, even with huffman compression on.12.04 = 1-2 bytes as float, 5 bytes as string.
459023 = 3 bytes as integer, 6 bytes as string.
And so forth. ResizeBitStream or InfiniteBitStream won't help you here.
What I would do if I were you, unless you plan to support the download-as-you-go modding, is to get rid of the datablock pack/unpack phase completely. That way you don't have to worry at all and your game will load faster.
We've done this in all our prototypes so far and it's not too complicated apart from a few ParticleEmitter changes in preload ().
#5
"We've done this in all our prototypes so far and it's not too complicated apart from a few ParticleEmitter changes in preload ()."
Specifically, what did you change?
04/14/2009 (11:15 am)
Stefan, can you elaborate on:"We've done this in all our prototypes so far and it's not too complicated apart from a few ParticleEmitter changes in preload ()."
Specifically, what did you change?
#6
I've just figured out what I could do do get my 2D arrays into script - use a loop of addFields, using dSprintf to give each a name like "myArray[x][y]", instead of using the addField with array size parameter.
But I've come to like this approach a litle more anyway - as long as I'm not making a player datablock that's several hundred lines, with all that stance data :P.
04/14/2009 (11:40 am)
Hmm... what are the disadvantages of getting rid of that completely? I'm guessing that by download-as-you-go modding, you mean letting the server-side user change datablock values and things, and clients getting sent correct data.I've just figured out what I could do do get my 2D arrays into script - use a loop of addFields, using dSprintf to give each a name like "myArray[x][y]", instead of using the addField with array size parameter.
But I've come to like this approach a litle more anyway - as long as I'm not making a player datablock that's several hundred lines, with all that stance data :P.
#7
Getting rid of the datablock transfer means (un)packData no longer is in use, and values that get sent from the server are no longer in need of packing from string ID's to data pointers.
Removing the checks in preload () and possibly onAdd () and using straight pointers to stuff instead of resolving via ID's is the way to go. Most other classes don't behave in this way.
04/14/2009 (3:00 pm)
Tim,Getting rid of the datablock transfer means (un)packData no longer is in use, and values that get sent from the server are no longer in need of packing from string ID's to data pointers.
Removing the checks in preload () and possibly onAdd () and using straight pointers to stuff instead of resolving via ID's is the way to go. Most other classes don't behave in this way.
Torque Owner Stefan Lundmark
Define healthy. ;)
There are downsides, and they aren't trivial. YMMV.
First off, your datablock group will be bigger which will lead to three things:
* There are more datablocks to send on mission load. 1 Datablock with 4 strings is faster to pack than 4 datablocks with 1 string.
* Unless I'm mistaken, the datablock ID number which is sent as part of most GameBase objects will require more bits per value to pack.
* Bloat in your scripts?
The reason Explosion contains several datablocks isn't to avoid a big datablock. Explosion shouldn't handle particles, ParticleEmitter should. There would lots of code duplication if Explosion did everything that ParticleEmitter, SFXSoundProfile, TSShape does.
You can also reuse those datablocks between other classes. It makes Torque more modular. Other than that, you should be safe. I sure wouldn't do it this way, and packing lots and lots of strings in a datablock usually isn't the right way to begin with.