Advanced Entity Subsystem - GOT part 1
by Tony Richards · 09/03/2009 (4:28 pm) · 8 comments
Torque's entity system, which in the engine source code starts with the class SimBase, is a purely hierarchical class structure. My next series of blogs will show off an entirely new entity subsystem that you can integrate with pretty much any of the C++ Torque game engines.
This entity subsystem is an integral part of the advanced game development middleware that I've been developing over the past... wow, 34 months now!
Mostly I have been concentrating on enhanced features and capabilities in support of my game Fractured Universe, but many of these features are useful for anyone wanting to make a game, whether it be an MMORPG or even something as simple as a Tower Defense clone.
When you use this middleware, you can still use Torque's rendering engine, audio layer, resource manager, etc, but I've replaced the network layer, the entity subsystem, the physics system, and the scripting subsystem and I've added some editors and a new object persistence layer.
What is a Game Entity System?
If you're interested in more technical details about Game Entities, I suggest reading this article by Mick West and this article by Britt L. Hannah. They're both fantastic articles and explore different frameworks for Game Entities.
What are the advantages of using Game Object Types for Entities?
The Game Object Type system (or GOT) that I've created is a lot easier to use than the existing system in stock Torque.
Whether you're a seasoned veteran game developer with tons of C++ skills or you're a fledgeling game developer barely able to modify some script, you'll benefit from the ease of use of the GOT system.
When you're writing a game with stock Torque and you want to add a new Game Entity, the first thing you do is write a couple of classes and stick it somewhere in the class hierarchy below GameBase and GameBaseData.
If it's a vehicle, you derive from Vehicle and VehicleData, or possibly it's some sort of specialized flying vehicle so you might want to derive from that.... etc.
As you're flushing out the details of these two new classes, there is a whole lot of glue code related to making it work with the editor, the datablock system, scripting and networking.
An advantage of using the GOT system is that you can create your own custom Game Entity without touching a single line of code.
Not C++... not even script. It's all done via a graphical editor.
When you're done, you can have the system be completely dynamic, or you can have the system generate C++ and script bindings.
During the design and development stages, you're likely going to want to keep everything dynamic, but once you start finalizing things you can switch to the high performance static system.
Once you've created your game entities, you don't have to worry about writing the code for replication, for the static initialization / datablocks, or even for persistence. You don't even have to worry about mapping your game entities to database tables.
All you need to do is start adding your game logic using script or NFA state machines... but I'll leave that for another blog.
Screenshots anyone? Here's a screenshot showing a portion of the Fractured Universe game design in the GOT editor.
As you can see, you use a graphical IDE to design the entities, enter the element types for each entity and enter the default values.
The default values are used much the same way as data blocks in stock Torque.
The element names are used to generate the C++ getter/setter methods as well as exposing them to script. I'm not showing it in this screenshot, but you can also assign privacy attributes so certain fields aren't available to the client-side script in order to help prevent cheating.
You can also assign persistence, scoping rules, replication rules and a customizable class factory for each entity.
The entity editor arranges the entities in a hierarchy, with each child entity inheriting all of the defaults from the parent.
You're able to add new elements as well as override default values and other properties in the child entities, making the system very flexible without putting too much burden on the game designer.
In my next blog, I'll go into a few more details about each of the options available in the persistence and replication properties.
Happy game development!
This entity subsystem is an integral part of the advanced game development middleware that I've been developing over the past... wow, 34 months now!
Mostly I have been concentrating on enhanced features and capabilities in support of my game Fractured Universe, but many of these features are useful for anyone wanting to make a game, whether it be an MMORPG or even something as simple as a Tower Defense clone.
When you use this middleware, you can still use Torque's rendering engine, audio layer, resource manager, etc, but I've replaced the network layer, the entity subsystem, the physics system, and the scripting subsystem and I've added some editors and a new object persistence layer.
What is a Game Entity System?
If you're interested in more technical details about Game Entities, I suggest reading this article by Mick West and this article by Britt L. Hannah. They're both fantastic articles and explore different frameworks for Game Entities.
What are the advantages of using Game Object Types for Entities?
The Game Object Type system (or GOT) that I've created is a lot easier to use than the existing system in stock Torque.
Whether you're a seasoned veteran game developer with tons of C++ skills or you're a fledgeling game developer barely able to modify some script, you'll benefit from the ease of use of the GOT system.
When you're writing a game with stock Torque and you want to add a new Game Entity, the first thing you do is write a couple of classes and stick it somewhere in the class hierarchy below GameBase and GameBaseData.
If it's a vehicle, you derive from Vehicle and VehicleData, or possibly it's some sort of specialized flying vehicle so you might want to derive from that.... etc.
As you're flushing out the details of these two new classes, there is a whole lot of glue code related to making it work with the editor, the datablock system, scripting and networking.
An advantage of using the GOT system is that you can create your own custom Game Entity without touching a single line of code.
Not C++... not even script. It's all done via a graphical editor.
When you're done, you can have the system be completely dynamic, or you can have the system generate C++ and script bindings.
During the design and development stages, you're likely going to want to keep everything dynamic, but once you start finalizing things you can switch to the high performance static system.
Once you've created your game entities, you don't have to worry about writing the code for replication, for the static initialization / datablocks, or even for persistence. You don't even have to worry about mapping your game entities to database tables.
All you need to do is start adding your game logic using script or NFA state machines... but I'll leave that for another blog.
Screenshots anyone? Here's a screenshot showing a portion of the Fractured Universe game design in the GOT editor.
As you can see, you use a graphical IDE to design the entities, enter the element types for each entity and enter the default values.
The default values are used much the same way as data blocks in stock Torque.
The element names are used to generate the C++ getter/setter methods as well as exposing them to script. I'm not showing it in this screenshot, but you can also assign privacy attributes so certain fields aren't available to the client-side script in order to help prevent cheating.
You can also assign persistence, scoping rules, replication rules and a customizable class factory for each entity.
The entity editor arranges the entities in a hierarchy, with each child entity inheriting all of the defaults from the parent.
You're able to add new elements as well as override default values and other properties in the child entities, making the system very flexible without putting too much burden on the game designer.
In my next blog, I'll go into a few more details about each of the options available in the persistence and replication properties.
Happy game development!
About the author
I am the founder of IndieZen.org, a website dedicated to the Indie 2.0 Revolution where a number of Indie game development studios and individuals collaborate and share a suite of custom built open source game development tools and middleware.
#3
09/03/2009 (6:18 pm)
Wow. I'm all questions, but I need to hear more before I can ask them! :) Sounds really interesting! 
#4
09/03/2009 (9:49 pm)
This sounds like black magic ;). I'm looking forward to hearing more about it. Especially the replaced networking and physics.
#5
09/03/2009 (11:15 pm)
Wow visual programming? Looking for more info.
#6
@Daniel - Black magic eh? "Any sufficiently advanced technology is indistinguishable from magic." - Arthur C. Clarke
I'd have to agree with you though.
If someone went back 3 years ago and told my former self and gave me the feature list of Zen Studio, I would've told them they were crazy and that it could never be done.... not in 10 years even with a huge team of programmers.
The project started out small enough, but it slowly snow-balled into an absolutely huge project. Not counting some of the experimental plugins that will probably never see the light of day, it's over half a million lines of code now.
The screenshot that I'm showing is the Game Builder plugin in Zen Studio. (there are other plugins in the works, including an Art Library, and some RPG editors for creating NPC dialogs, Quests, etc).
The interesting thing is that this project has been a "chicken or egg" project...
Zen Studio's database persistence layer was generated using Zen Studio!
Now that's what I call magic...
But, it's really not as magical as it sounds.
As far as Torque's networking and physics, there's really nothing wrong with the stock Torque implementation as-is and I didn't really improve anything as far as the implementation goes. The only thing I did was rearrange them so that they're easier to use for non-programmers. That took a bit of a re-write, and it actually has slightly higher overhead than Torque's streamlined stock implementation, but the trade-off to make it easier to use was worth it.
I replaced the physics with physics framework a lot like the Physics Abstraction Layer project. I'm using the same plugin system as Zen Studio to load the physics plugin.
I'm also using a system I'm calling Physics Behaviors, but it's really just a glorified call-back system. I have a several pre-built behaviors for handling things like gravity and simple avatar walking, jumping, flying, etc.
You know the portion of code in the Player, FlyingVehicle, etc that handles the physics in Toruqe? Essentially I've just pulled that out of the Entity classes and turned it into callbacks. Nothing magic there, but it makes it so people can shared "behavior" plugins.
The advantage is the integration with the entity editor and the ease of using pre-built behavior plugins. This may not be much of an advantage for skilled C++ programmers, but for the average Torque user (especially the hobbyists) it's a huge benefit.
I won't go into the gory details, but the plugins not only integrate with Torque and this entity system, but they're also used in the editors so you click the mouse to select the behaviors that a game entity should use.
The same is true of replication. I'm using a high performance cross platform C++ network library (Boost ASIO) for the lower socket layer, but I'm using plugins for the replicator and scopator.
The replication plugins that will be available immediately are:
* High Fidelity, which is a whole lot like Torque's ghosting model and I stole a whole lot of ideas out of it :P This replication model is perfect for FPS and Racing type games.
* Low Fidelity, which is my own custom system with lazier interpolation rules for games that don't require the avatar's position to be as accurate (this is awesome for RPG games)
* Pathing and Flocking, which is another custom replication system generally useful for "click to move" type games... adventure games, RTS games, and even some RPG games.
A scopator is an implementation of scoping rules. Stock Torque uses a limited set of scoping rules... some things are never scoped, some things are always scoped, and other things are only scoped if they're in the player's camera. Most of the time this is good enough, but sometimes it's not.
The replication system that I'm including with the MMO kit I'm creating has a scopator that's similar in nature to Torque's stock scopator, but it also includes two others that are closer in design to the RTS Starter Kit and another one that I designed specifically for use in MMO games.
Again, the replication code itself isn't any better than stock Torque networking (in fact, it probably has a slightly higher overhead).
It's the design and the use of plugins for the replication model and scoping rules that makes it better... again, probably not better for your skilled C++ programmer that never had any difficulty with Torque in the first place, but for everyone else that has had difficulty, it's yet another huge benefit.
Not to mention the fact that the code's generated for you and even if you're a highly skilled C++ programmer, it still saves you a lot of time. :P
09/04/2009 (12:10 pm)
@Daniel - Black magic eh? "Any sufficiently advanced technology is indistinguishable from magic." - Arthur C. Clarke
I'd have to agree with you though.
If someone went back 3 years ago and told my former self and gave me the feature list of Zen Studio, I would've told them they were crazy and that it could never be done.... not in 10 years even with a huge team of programmers.
The project started out small enough, but it slowly snow-balled into an absolutely huge project. Not counting some of the experimental plugins that will probably never see the light of day, it's over half a million lines of code now.
The screenshot that I'm showing is the Game Builder plugin in Zen Studio. (there are other plugins in the works, including an Art Library, and some RPG editors for creating NPC dialogs, Quests, etc).
The interesting thing is that this project has been a "chicken or egg" project...
Zen Studio's database persistence layer was generated using Zen Studio!
Now that's what I call magic...
But, it's really not as magical as it sounds.
As far as Torque's networking and physics, there's really nothing wrong with the stock Torque implementation as-is and I didn't really improve anything as far as the implementation goes. The only thing I did was rearrange them so that they're easier to use for non-programmers. That took a bit of a re-write, and it actually has slightly higher overhead than Torque's streamlined stock implementation, but the trade-off to make it easier to use was worth it.
I replaced the physics with physics framework a lot like the Physics Abstraction Layer project. I'm using the same plugin system as Zen Studio to load the physics plugin.
I'm also using a system I'm calling Physics Behaviors, but it's really just a glorified call-back system. I have a several pre-built behaviors for handling things like gravity and simple avatar walking, jumping, flying, etc.
You know the portion of code in the Player, FlyingVehicle, etc that handles the physics in Toruqe? Essentially I've just pulled that out of the Entity classes and turned it into callbacks. Nothing magic there, but it makes it so people can shared "behavior" plugins.
The advantage is the integration with the entity editor and the ease of using pre-built behavior plugins. This may not be much of an advantage for skilled C++ programmers, but for the average Torque user (especially the hobbyists) it's a huge benefit.
I won't go into the gory details, but the plugins not only integrate with Torque and this entity system, but they're also used in the editors so you click the mouse to select the behaviors that a game entity should use.
The same is true of replication. I'm using a high performance cross platform C++ network library (Boost ASIO) for the lower socket layer, but I'm using plugins for the replicator and scopator.
The replication plugins that will be available immediately are:
* High Fidelity, which is a whole lot like Torque's ghosting model and I stole a whole lot of ideas out of it :P This replication model is perfect for FPS and Racing type games.
* Low Fidelity, which is my own custom system with lazier interpolation rules for games that don't require the avatar's position to be as accurate (this is awesome for RPG games)
* Pathing and Flocking, which is another custom replication system generally useful for "click to move" type games... adventure games, RTS games, and even some RPG games.
A scopator is an implementation of scoping rules. Stock Torque uses a limited set of scoping rules... some things are never scoped, some things are always scoped, and other things are only scoped if they're in the player's camera. Most of the time this is good enough, but sometimes it's not.
The replication system that I'm including with the MMO kit I'm creating has a scopator that's similar in nature to Torque's stock scopator, but it also includes two others that are closer in design to the RTS Starter Kit and another one that I designed specifically for use in MMO games.
Again, the replication code itself isn't any better than stock Torque networking (in fact, it probably has a slightly higher overhead).
It's the design and the use of plugins for the replication model and scoping rules that makes it better... again, probably not better for your skilled C++ programmer that never had any difficulty with Torque in the first place, but for everyone else that has had difficulty, it's yet another huge benefit.
Not to mention the fact that the code's generated for you and even if you're a highly skilled C++ programmer, it still saves you a lot of time. :P
#7
The concept has been around for many years in the enterprise world. Take a look at products like IBM's Rational tools and Apple's Data Modeling tool that comes with XCode.
I've just taken it further by adding some features to handle replication and entity designers.
@CSMP - "Good Job speeding up your workflow!"
I can't help but laughing at myself over that statement.
I had an awesome prototype of Fractured Universe finished in under 4 months... and that was over 2.5 years ago. I got a bit distracted with writing my own tools. Had I not gotten distracted and had I continued at a reasonable pace, quite likely I would have been finished by now... in fact, I probably would have been finished more than a year ago.
But really you're right. Although it won't net out to be a faster delivery of Fractured Universe, it does significantly improve the workflow for most games, so I can start pumping games out faster than most people, and a few years from now when I've finished a dozen or more games, I'll be a lot better off than if I hadn't made the tools.
But really, the plan isn't for me to be the only person to benefit from this. The plan is to make it so that all Indies can use it.
I think the best benefit of the Torque integration would be if I worked out a deal with Garage Games and distributed this with the "Lite" version of T3D. Since you don't need the source code of Torque to make it work (you just need T3D in a .dll), I think it'd make a fantastic 3d "Game Builder" platform while still keeping the price way down.
But who knows... possibly the GG skunk-works team already has something like this in the works :P
09/04/2009 (12:11 pm)
@Sei - It's not really visual "programming". It's visual data modeling.The concept has been around for many years in the enterprise world. Take a look at products like IBM's Rational tools and Apple's Data Modeling tool that comes with XCode.
I've just taken it further by adding some features to handle replication and entity designers.
@CSMP - "Good Job speeding up your workflow!"
I can't help but laughing at myself over that statement.
I had an awesome prototype of Fractured Universe finished in under 4 months... and that was over 2.5 years ago. I got a bit distracted with writing my own tools. Had I not gotten distracted and had I continued at a reasonable pace, quite likely I would have been finished by now... in fact, I probably would have been finished more than a year ago.
But really you're right. Although it won't net out to be a faster delivery of Fractured Universe, it does significantly improve the workflow for most games, so I can start pumping games out faster than most people, and a few years from now when I've finished a dozen or more games, I'll be a lot better off than if I hadn't made the tools.
But really, the plan isn't for me to be the only person to benefit from this. The plan is to make it so that all Indies can use it.
I think the best benefit of the Torque integration would be if I worked out a deal with Garage Games and distributed this with the "Lite" version of T3D. Since you don't need the source code of Torque to make it work (you just need T3D in a .dll), I think it'd make a fantastic 3d "Game Builder" platform while still keeping the price way down.
But who knows... possibly the GG skunk-works team already has something like this in the works :P
#8
09/06/2009 (4:05 pm)
Great Tony! :-D 
Torque Owner CSMP
MP Studios