Refactoring game entities with components
by Daniel Buckmaster · in General Discussion · 08/06/2012 (3:01 am) · 19 replies
Don't get to excited just yet. I read a very interesting article just now. It's about the 'components' design pattern that modern game engines seem to be heading towards.
Article!
Some selected quotes:
The idea of components is that an object, instead of being a single class, is just a collection of disparate small objects, each with their own particular data and methods. For example, a Projectile would be comprised of a TSShape component, a networking component, and a simple physics component that just moves it forward at some constant speed (with optional gravity).
Of course, components can't really be totally separate. For example, both TSShape and a physics component would need to know the position of the object in the scene. (Or maybe you could refactor it so that the TSShape component was entirely position-less, just providing rendering. Or, as suggested in the article, 'position' itself is a separate component. Which is pretty groovy.)
From what I've read of its documentation, Unity uses this system, as do several HTML5 game engines I've seen. I can't comment on the other big guns of the game industry like Crytek or Unreal, but it seems like a very future-proofed design pattern.
How does this relate to Torque? I dunno. Aside from the fact that Torque doesn't do this, I just wanted to generate a discussion on the merits (or otherwise) of this pattern, maybe even with the intent of thinking about the feasibility of refactoring Torque's core gameplay classes. Which is insane, I know. Could be fun, though.
Article!
Some selected quotes:
Quote:Even fairly simple objects such as rocks or grenades can end up with a large amount of additional functionality (and associated member variables, and possibly unnecessary execution of member functions). Often, the traditional game object hierarchy ends up creating the type of object known as "the blob". The blob is a classic "anti-pattern" which manifests as a huge single class (or a specific branch of a class hierarchy) with a large amount of complex interwoven functionality.Sounds like ShapeBase to me. Though Projectile, for example, sidesteps ShapeBase by deriving directly from SceneObject, that means:
Quote:Programmers often duplicate code to mirror functionality already implemented in a different object. Eventually messy re-factoring is required by re-structuring the class hierarchy to move and combine functionality.Projectile has to duplicate rendering code, for example.
The idea of components is that an object, instead of being a single class, is just a collection of disparate small objects, each with their own particular data and methods. For example, a Projectile would be comprised of a TSShape component, a networking component, and a simple physics component that just moves it forward at some constant speed (with optional gravity).
Quote:The component approach, which is gaining more acceptance in current game development, is one of separating the functionality into individual components that are mostly independent of one another.
Of course, components can't really be totally separate. For example, both TSShape and a physics component would need to know the position of the object in the scene. (Or maybe you could refactor it so that the TSShape component was entirely position-less, just providing rendering. Or, as suggested in the article, 'position' itself is a separate component. Which is pretty groovy.)
From what I've read of its documentation, Unity uses this system, as do several HTML5 game engines I've seen. I can't comment on the other big guns of the game industry like Crytek or Unreal, but it seems like a very future-proofed design pattern.
How does this relate to Torque? I dunno. Aside from the fact that Torque doesn't do this, I just wanted to generate a discussion on the merits (or otherwise) of this pattern, maybe even with the intent of thinking about the feasibility of refactoring Torque's core gameplay classes. Which is insane, I know. Could be fun, though.
About the author
Studying mechatronic engineering and computer science at the University of Sydney. Game development is probably my most time-consuming hobby!
#2
I keep toying with the idea of trying to do something with it, as the engine is starting to feel outdated and nasty. Every project involves fighting against the ShapeBase bloat or all of the unnecessary spaghetti that the Player class carries around.
With T3D, even subclassing GameBase to create new classes is now a pain, as the lighting system expects objects that are lit/cast shadows to be derived from ShapeBase. Being able to create new classes from an assembly of parts would be a much nicer way to do things.
TGB has elements of a component design with its behaviors, although it doesn't take it to the same level as Unity, where everything, even a transform is a component.
08/06/2012 (5:51 am)
T3D has the framework for a component system in place in engine\component. This was started several years back, and was never developed further. I keep toying with the idea of trying to do something with it, as the engine is starting to feel outdated and nasty. Every project involves fighting against the ShapeBase bloat or all of the unnecessary spaghetti that the Player class carries around.
With T3D, even subclassing GameBase to create new classes is now a pain, as the lighting system expects objects that are lit/cast shadows to be derived from ShapeBase. Being able to create new classes from an assembly of parts would be a much nicer way to do things.
TGB has elements of a component design with its behaviors, although it doesn't take it to the same level as Unity, where everything, even a transform is a component.
#3
You would have to send the 'assembly' of components for an object from the server to the client in order to instantiate that object client side. Then, would each component handle its own networking? So would a transform component handle networking its position and rotation? If that was the case, then you would need different kinds of transform components with different networking rules. e.g. Player networks position and z-rotation each tick. A projectile only networks its position and rotation once, when it is created.............
tricky.
08/06/2012 (6:37 am)
One thing I struggle with understanding is how you would handle the networking with such a system.You would have to send the 'assembly' of components for an object from the server to the client in order to instantiate that object client side. Then, would each component handle its own networking? So would a transform component handle networking its position and rotation? If that was the case, then you would need different kinds of transform components with different networking rules. e.g. Player networks position and z-rotation each tick. A projectile only networks its position and rotation once, when it is created.............
tricky.
#4
The problem with this entire line of thought, as the guy in the article mentions, is that refactoring the engine into modules would be one of those changes that doesn't initially do anything. Your end goal is to have everything work exactly the way it did before, at least from the user's POV.
EDIT: More resources:
Forum post about Dungeon Seige's engine being refactored into components.
In-depth introduction to the component pattern in games.
08/07/2012 (2:38 am)
Quote:Then, would each component handle its own networking?I imagine that each component might have, for example, packUpdate and unpackUpdate methods. Then, yes, a Player's transform component might be different to an Item's (which is a bad example, since Item's code is basically a butchered version of Player). I haven't read up much on component-based systems; I assume lots of problems like this have been solved. Another one brought up in the article is the issue of synchronisation - what if one object's physics component updates before its animation, and vice versa for another object?
Quote:It doesn't seem completely insane to doI beg to differ ;P. You'd be basically rewriting most of the engine, I think. Gameplay code, at least. The low-level GFX/SFX/console stuff could stay, but even, for example, scene traversal would probably have to be updated. Not to mention the way the console interacts with components.
Quote:T3D has the framework for a component system in place in enginecomponent.Holy moley, those comments mention Juggernaut! Thanks for the tip, I had no idea that existed. Can't really make head or tail of the system, though, without any high-level documentation. Good to know GG were thinking ahead, I guess.
The problem with this entire line of thought, as the guy in the article mentions, is that refactoring the engine into modules would be one of those changes that doesn't initially do anything. Your end goal is to have everything work exactly the way it did before, at least from the user's POV.
EDIT: More resources:
Forum post about Dungeon Seige's engine being refactored into components.
In-depth introduction to the component pattern in games.
#5
Not promoting the competition here, but if you haven't already done it, I would recommend downloading Un*ty and just taking a look at how their system works. It's pretty slick the way you can create stuff just by plugging together components and writing some small script glue.
08/07/2012 (6:54 am)
@Daniel, Not promoting the competition here, but if you haven't already done it, I would recommend downloading Un*ty and just taking a look at how their system works. It's pretty slick the way you can create stuff just by plugging together components and writing some small script glue.
#6
One more comment on networking. You could probably get into situations where an object could actually be composed of different components on the client and server. For example, an AIPlayer on the server may not need a rendering component, but it would need an AI component (assume AI is all done server-side). On the client, it would need a rendering component, sound component, etc.
08/07/2012 (4:21 pm)
I've played around with it a bit, and that experience was what prompted me to research components a bit more.One more comment on networking. You could probably get into situations where an object could actually be composed of different components on the client and server. For example, an AIPlayer on the server may not need a rendering component, but it would need an AI component (assume AI is all done server-side). On the client, it would need a rendering component, sound component, etc.
#7
1) Only allow one component of a type/class on an object at a time. It will make your life much simpler and your coding far more sane.
2) Don't let components talk directly to each other if at all possible. Instead, have them send signals/notifications at either the entity level (heard only by sibling components) or broadcast to the entire game state (heard by all entities/components listening for that signal).
I highly recommend not trying to write your own C++ signals system. It is hard to do and prone to errors. Fortunately, a little Googling will turn up several good options. At this point I can't recommend one since I relied on the NSNotification's when doing Objective-C/iOS programming and am currently using a nice AS3 system in Flash.
3) Templates, for both entities (a list of components) and components, are your friends. Templates should be hierarchical.
08/14/2012 (1:03 pm)
A few tips I can share after working with some great component systems:1) Only allow one component of a type/class on an object at a time. It will make your life much simpler and your coding far more sane.
2) Don't let components talk directly to each other if at all possible. Instead, have them send signals/notifications at either the entity level (heard only by sibling components) or broadcast to the entire game state (heard by all entities/components listening for that signal).
I highly recommend not trying to write your own C++ signals system. It is hard to do and prone to errors. Fortunately, a little Googling will turn up several good options. At this point I can't recommend one since I relied on the NSNotification's when doing Objective-C/iOS programming and am currently using a nice AS3 system in Flash.
3) Templates, for both entities (a list of components) and components, are your friends. Templates should be hierarchical.
#8
T3D has a signal/slot mechanism built in, along with a global messaging system, I wonder if they would be up to the task required for this kind of thing.
I just stumbled across this blog about a component based solution for torque. Looks good, shame it appears to have died :(
Also, Game Programming Gems 6 has a chapter about this too.
08/15/2012 (10:52 am)
Some good points there. It makes sense not to allow multiple physics components etc on a single object.T3D has a signal/slot mechanism built in, along with a global messaging system, I wonder if they would be up to the task required for this kind of thing.
I just stumbled across this blog about a component based solution for torque. Looks good, shame it appears to have died :(
Also, Game Programming Gems 6 has a chapter about this too.
#9
The Zen Studio from that blog can be found there, but I don't think it have any direct integration with T3D included.
Also the blog was AFAIK talking about TGEA, but maybe we could contact the developer and ask about the status of the project and if we could get it so we could port it to T3D and expand it?
Unfortunately I wont have the time for such a project :P
08/15/2012 (11:03 am)
Found it's websiteThe Zen Studio from that blog can be found there, but I don't think it have any direct integration with T3D included.
Also the blog was AFAIK talking about TGEA, but maybe we could contact the developer and ask about the status of the project and if we could get it so we could port it to T3D and expand it?
Unfortunately I wont have the time for such a project :P
#10
A small contribution based on my experience of the ones I used / implemented -
First, I totally agree with Matt Fairfax ^^^^
Also, I find it useful to be able to build components with data coming from multiple places ; usually, I end up splitting my components into 1. template data (shared by all the instances), 2. instance data (specific to each individual GameObject). (and potentially a way to locally override template settings, but that's not used very often)
1. Template data comes from description files of the GameObject entities (these would contain shared attributes for display components, physics components, behaviour components, navigation component, etc.).
2. Instance data comes from an external editor (these would be the positions of the entities, their names for scripting, any kind of attributes that can be set on a per-instance basis, etc.). Basically, everything that cannot be shared and placed in the "template" definition, but is required to initialise the entities properly
Therefore a GameObject description is split into its shared parts and its unique ones. Some components are "template" only, some are "instance" only, and some need a bit of both ; all depending on how you structure your entities, and how much customisation you require for each logic "block"
It's also fairly easy to move data from template<>instance, to expose more or less of the settings through editors, etc. depending on the needs of the game.
At runtime, the template and the instance "partial" data blocks are assembled into a higher level "component" that provides the GameObject with everything required to handle its own logic (inc. data that it neither template nor instance, i.e. runtime date, e.g. current set of values for things that evolve during the game), and process messages from other components.
In the end it's a very flexible system that lets you define your entities in a very "descriptive" way, simply by specifying what components you need on each entity. Plus it "flattens" the entire hierarchy, making it possible to combine any components as long as their logic is not conflicting.
It also means that with the right tools (or some simple hand-edited XML description files), game designers can come up very quickly with the logic they want for their entities (assuming the pool of available components provides everything needed), instead of having to go to a programmer for every variant or combination of logic they need.
E.g. in a dungeon crawler game I'm working on, we have monsters and traps (typical...) ; the navigation components are totally decoupled from the others, therefore the designer came up with a moving trap that chases the player, simply by attaching the appropriate component to an existing trap template.
That's what designers want, the ability to experiment and iterate to improve the overall gameplay. It really helps not having bottlenecks on the programming side, for these things.
Obviously, components need to be "atomic" and designed to be able to work with each other for the whole system to produce something useful.
This is very good alternative to hierarchy based GameObject frameworks. I strongly recommend using one :) (I haven't used the Torque one though, so I can't really comment on it, I've implemented my own)
Regards
08/16/2012 (8:02 am)
Interesting discussion ! I love component based "GameObject" frameworks :)A small contribution based on my experience of the ones I used / implemented -
First, I totally agree with Matt Fairfax ^^^^
Also, I find it useful to be able to build components with data coming from multiple places ; usually, I end up splitting my components into 1. template data (shared by all the instances), 2. instance data (specific to each individual GameObject). (and potentially a way to locally override template settings, but that's not used very often)
1. Template data comes from description files of the GameObject entities (these would contain shared attributes for display components, physics components, behaviour components, navigation component, etc.).
2. Instance data comes from an external editor (these would be the positions of the entities, their names for scripting, any kind of attributes that can be set on a per-instance basis, etc.). Basically, everything that cannot be shared and placed in the "template" definition, but is required to initialise the entities properly
Therefore a GameObject description is split into its shared parts and its unique ones. Some components are "template" only, some are "instance" only, and some need a bit of both ; all depending on how you structure your entities, and how much customisation you require for each logic "block"
It's also fairly easy to move data from template<>instance, to expose more or less of the settings through editors, etc. depending on the needs of the game.
At runtime, the template and the instance "partial" data blocks are assembled into a higher level "component" that provides the GameObject with everything required to handle its own logic (inc. data that it neither template nor instance, i.e. runtime date, e.g. current set of values for things that evolve during the game), and process messages from other components.
In the end it's a very flexible system that lets you define your entities in a very "descriptive" way, simply by specifying what components you need on each entity. Plus it "flattens" the entire hierarchy, making it possible to combine any components as long as their logic is not conflicting.
It also means that with the right tools (or some simple hand-edited XML description files), game designers can come up very quickly with the logic they want for their entities (assuming the pool of available components provides everything needed), instead of having to go to a programmer for every variant or combination of logic they need.
E.g. in a dungeon crawler game I'm working on, we have monsters and traps (typical...) ; the navigation components are totally decoupled from the others, therefore the designer came up with a moving trap that chases the player, simply by attaching the appropriate component to an existing trap template.
That's what designers want, the ability to experiment and iterate to improve the overall gameplay. It really helps not having bottlenecks on the programming side, for these things.
Obviously, components need to be "atomic" and designed to be able to work with each other for the whole system to produce something useful.
This is very good alternative to hierarchy based GameObject frameworks. I strongly recommend using one :) (I haven't used the Torque one though, so I can't really comment on it, I've implemented my own)
Regards
#11
Something like this could be done in CE and would not need to be in a branch. It could be a different class based upon say sceneobject or even simobject. That might even be too high a level, I dunno. Anyway, it could be a completely different set of classes that do not conflict at all.
This is something I would put some time into if other people wanted to work on this. I really like the idea of being able to make a game networkable just by adding a component to an object. Of course you would need a mechanism to define what data needs to be networked for each objects, but that could be done through a small database for each object. The networking component could query the host object for what data is important for state between clients.
I have experience with xml and xsl if that helps. If someone with some experience on this wanted to assign tasks I would be willing to do some of the code work. I just don't have the time to research building components right now. I have some experience with deriving objects from GameBase as well.
@Raphael,
Would you be willing to direct this since you have that experience? Again, I would do the legwork and have you review it.
08/16/2012 (7:41 pm)
This is something I felt TGE needed years ago. I could not describe in words how it would work, but I felt objects should be able to be constructed. So the term is 'components' in today's game jargon.Something like this could be done in CE and would not need to be in a branch. It could be a different class based upon say sceneobject or even simobject. That might even be too high a level, I dunno. Anyway, it could be a completely different set of classes that do not conflict at all.
This is something I would put some time into if other people wanted to work on this. I really like the idea of being able to make a game networkable just by adding a component to an object. Of course you would need a mechanism to define what data needs to be networked for each objects, but that could be done through a small database for each object. The networking component could query the host object for what data is important for state between clients.
I have experience with xml and xsl if that helps. If someone with some experience on this wanted to assign tasks I would be willing to do some of the code work. I just don't have the time to research building components right now. I have some experience with deriving objects from GameBase as well.
@Raphael,
Would you be willing to direct this since you have that experience? Again, I would do the legwork and have you review it.
#12
@Frank, this is something that I've been dreaming about for several years as a way of breaking away from the horrible bloated hierarchy that torque relies on. I've never been able to get a firm handle on how to implement it. I think that something like this is essential to bring T3D up to date.
If you decide to pursue this further, I'd like to get in on the act also :)
08/17/2012 (1:41 am)
Great blog here discussing component based entity systems. Also links to a wiki and some source code examples.@Frank, this is something that I've been dreaming about for several years as a way of breaking away from the horrible bloated hierarchy that torque relies on. I've never been able to get a firm handle on how to implement it. I think that something like this is essential to bring T3D up to date.
If you decide to pursue this further, I'd like to get in on the act also :)
#13
08/17/2012 (1:50 am)
I think I could take some of the job on my shoulders aswell, but definetly not alot, I wont have the time for it. But I could contribute with the smaller tasks!
#14
Nevertheless, I was thinking of maybe packaging my entity/component framework into a "Torque resources" (some doc, UML class diagrams, and the core source code) ; it's far from perfect, and I'm aware of it (doesn't cope nicely with multithreading of components logic, the messaging system is also a bit weak, etc. It's got some good mechanisms, still), but it's suitable for small games and might be a useful starting point for someone else to use.
Anyway, this thread seems like a good opportunity to start at the "requirements" stage again, and design a solution that works for "us". Unless someone can point to an existing resource that does exactly that ; most available "component framework"-related resources appear to be high level articles, though, leaving the design and implementation of the actual solution to the reader...
I'll try to make a list of the requirements I considered when designing the architecture of my small framework, solutions I found to problems I encountered, and weakness I'm aware of ; that might be useful input :-?
08/17/2012 (6:36 am)
@Frank Carney : time would be difficult to find me at the moment, but I could definitely have "some" level of inputNevertheless, I was thinking of maybe packaging my entity/component framework into a "Torque resources" (some doc, UML class diagrams, and the core source code) ; it's far from perfect, and I'm aware of it (doesn't cope nicely with multithreading of components logic, the messaging system is also a bit weak, etc. It's got some good mechanisms, still), but it's suitable for small games and might be a useful starting point for someone else to use.
Anyway, this thread seems like a good opportunity to start at the "requirements" stage again, and design a solution that works for "us". Unless someone can point to an existing resource that does exactly that ; most available "component framework"-related resources appear to be high level articles, though, leaving the design and implementation of the actual solution to the reader...
I'll try to make a list of the requirements I considered when designing the architecture of my small framework, solutions I found to problems I encountered, and weakness I'm aware of ; that might be useful input :-?
#15
@Guy and Lukas,
Okay, I will work on this in a 2 to 4 weeks. I want to finish exploring the terrain generation code that is being discussed here: www.garagegames.com/community/forums/viewthread/131070
@Guy,
Thanks for the link. That will give me something to look at to get my head around this. I think the hardest part will be the networking.
If anyone has any ideas for good ways to build components please chime in. I think looking at the problem is many different ways is a good thing.
08/17/2012 (6:38 am)
If the components could be done at the script or xml level that would be ideal. It would allow GUIs to be built to construct the entities in a WYSIWYG interface. Actually just getting this to script calls would be fine. You can build the script calls from xml if you really want to.@Guy and Lukas,
Okay, I will work on this in a 2 to 4 weeks. I want to finish exploring the terrain generation code that is being discussed here: www.garagegames.com/community/forums/viewthread/131070
@Guy,
Thanks for the link. That will give me something to look at to get my head around this. I think the hardest part will be the networking.
If anyone has any ideas for good ways to build components please chime in. I think looking at the problem is many different ways is a good thing.
#16
Just to give a quick overview of what I have ATM, a typical "template" definition in my system looks like that (this is part of a manually edited file) :
This is a very simple one, a key lock wall element with a specific style. More complex objects have dozens of components (e.g. monster, with health stats, combat logic, navigation logic, target selection component, animation, etc.).
For this template to be reusable, it does not specify it's "intrinsic" data, i.e. its position in the world, the target object for the messages to be sent, etc. These come from the instance data, generated by an editor I made in parallel ; the instance data looks like :
At runtime, I parse the database of template definitions, parse the level file with the instance data, assemble the "partial" data blocks (template + instance) into full components, and assign them to a GameObject. The GameObject interface is extremely high level and entirely generic ; all it can do, pretty much, is Advance and Receive messages. It delegates to all its active components, who are in charge of dealing with their own logic.
As long as the components logic don't conflict with each other, and that implicit dependencies are satisfied (e.g. a navigation component requires the object to support a position component, an animation component requires the object to support a display or model component, etc.), they can be arranged by designers the way they like. Components query data from each other, and post messages to their parent GameObject to trigger events or send requests to other components.
What's great is that is also give a generalised form of Strategy design pattern for all your logic blocks, therefore it's very easy to extend and recombine to create the desired effect.
08/17/2012 (7:36 am)
@Frank : definitely, XML is the way to go ! It's easy to hand-edit the descriptions, designers are familiar with it so they can fiddle with values and components, an editor can be added later if the cost/productivity ratio is worth it, etc. And it's also possible to annotate the XML to generate XAML/WPF UI controls automatically.Just to give a quick overview of what I have ATM, a typical "template" definition in my system looks like that (this is part of a manually edited file) :
<Template name="Keylock_Iron">
<Editor type="Mount"/>
<Component name="MountPosition"/>
<Component name="LockInteraction">
<TemplateData enabled="1" activateMsg="Toggle" mismatchMsg="None" consumeKey="1"/>
</Component>
<Component name="BasicDisplay">
<TemplateData imageMap="WallLock_Iron"/>
</Component>
</Template>This is a very simple one, a key lock wall element with a specific style. More complex objects have dozens of components (e.g. monster, with health stats, combat logic, navigation logic, target selection component, animation, etc.).
For this template to be reusable, it does not specify it's "intrinsic" data, i.e. its position in the world, the target object for the messages to be sent, etc. These come from the instance data, generated by an editor I made in parallel ; the instance data looks like :
<GameObject templateName="Keylock_Iron" instanceName="">
<Component name="MountPosition">
<InstanceData x="17" y="12" wall="0" slot="Center" />
</Component>
<Component name="LockInteraction">
<InstanceData keyName="key02" targetName="door07" />
</Component>
</GameObject>At runtime, I parse the database of template definitions, parse the level file with the instance data, assemble the "partial" data blocks (template + instance) into full components, and assign them to a GameObject. The GameObject interface is extremely high level and entirely generic ; all it can do, pretty much, is Advance and Receive messages. It delegates to all its active components, who are in charge of dealing with their own logic.
As long as the components logic don't conflict with each other, and that implicit dependencies are satisfied (e.g. a navigation component requires the object to support a position component, an animation component requires the object to support a display or model component, etc.), they can be arranged by designers the way they like. Components query data from each other, and post messages to their parent GameObject to trigger events or send requests to other components.
What's great is that is also give a generalised form of Strategy design pattern for all your logic blocks, therefore it's very easy to extend and recombine to create the desired effect.
#17
Yes, it would be very useful. There is nothing like the experience of someone who has been there before. So please do share.
08/17/2012 (7:36 am)
@Raphael,Yes, it would be very useful. There is nothing like the experience of someone who has been there before. So please do share.
#18
There are definetely examples out there of complex framework that allow for even greater flexibility, script interface, etc.
08/17/2012 (7:40 am)
Btw, I'm not pretending this is *the* solution, it's just a light-weight one I put together quickly for a small prototype. But it works for me :)There are definetely examples out there of complex framework that allow for even greater flexibility, script interface, etc.
Torque Owner Lukas Joergensen
WinterLeaf Entertainment