Entity/Component System Released!
by Jeff Raab · 07/18/2014 (6:12 pm) · 45 comments
Hey all!
github.com/Areloch/Torque3D/tree/EntityComponent_Experimental
BAM.
Wait, what? You're releasing it? With no community purchase campaign? Are you sick? Dying? IS THIS YOUR FINAL WILL AND TESTAMENT?!
What? No, that's stupid.
Whew, ok, yeah, that got a little dark there. But seriously, what's going on?
It pretty much boils down to the fact that no matter WHAT I did, it wouldn't be polished enough for me to be especially happy with a "release". This was compounded by the last week or so where I'd been beating my head on a wall trying to fix some "last bugs"(I've been doing that for a month now), and just could not, for the life of me, figure it out. I know I CAN, but it was taking far more time than it should, and I was making me hate myself over it.
That, and honestly, nothing I could charge in the Community Purchase campaign could ever REALLY make up the...what, like, 6 months of effort that's gone into this now?
So hey, whatever.
So...what, are you done with this then?
Oh God no. I plan to use this system in the games I work on, so even if no one ever touches the repo, I'll keep updating it.
This is pretty much just forcing me to let it go and get other people who are smarter than me working on it to fix crap I failed to.
The obvious endgoal is it gets polished to a mirror sheen, and then it gets rolled into the T3D development branch, and then eventually master. But it's got a lotta work to go before that's happening and I figure, why not start now.
So my repo could be considered the definitive Torque3D Entity/Component branch that we'll all(hopefully) collectively polish until we feel comfortable about rolling it into an official branch for proper testing pre-integration.
However, it's in a very, very rough state now. Tons and tons and tons of chaff from self comments, testing code and as-we-go rewrites of entire systems and implementations clog the current implementation. To say nothing of experimental behaviors I didn't get really working still sitting on my harddrive for later.
As such, I have no doubts, at all in my mind, that it'll be kinda crap to navigate or for people to really work with. But you know what? That's OK.
How is that OK? That's passing the buck, isn't it?
Shoosh, you. No, it's OK, because I'm throwing down an open invitation to talk to me here on the forums, in chat, or via email at any time for clarification, suggestions, or ideas.
This is a community project, and while I've got a pretty concrete idea of where it's going, the whole point is we'd work on it together to make it perfect.
Similarly, if any of you guys want to make your own components for stuff, or convert existing behavior in the engine/your projects into components/behaviors, feel free to ask. I'm absolutely happy to help brainstorm how to component-ize everything.
If you don't know what you want to componentize, but wanna do something, like said, I'm sitting on half-implemented stuff I think could be a pretty big deal if it was made to work right. So by all means, please fire me a line.
So it works out of the box then?
More or less, yeah.
In the branch is a new template. The EntityComponent template.
When making a new project with it, you'll have to add the missing folders/files in the engine/component directory as I haven't fixed the project template stuff yet, but after that it should compile and run fine.
The template is based on Dan Buckmaster's Torque3D Barebones template, only this has menus, editors and server/client functionality implemented as well. So it's a more "game ready" template.
Currently, it boots straight into the mission, but you can re-enable it going to the main menu by tweaking what it loads to in the main.cs.
(Again, if you have any questions, fire them at me)
So all that effort, for free?
More or less, yeah.
I mean, if you've got some extra cash and want to throw it at me? Sure, why not.
I've got a paypal, here:
www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=N3L6L64NGSDM8
Feel free to toss something my way if you think I deserve it.
With that said, grab the repo, balk at my coding atrocities, and once you get over the whiplash, lets get Torque'ing :D
-Jeff out
github.com/Areloch/Torque3D/tree/EntityComponent_Experimental
BAM.
Wait, what? You're releasing it? With no community purchase campaign? Are you sick? Dying? IS THIS YOUR FINAL WILL AND TESTAMENT?!
What? No, that's stupid.
Whew, ok, yeah, that got a little dark there. But seriously, what's going on?
It pretty much boils down to the fact that no matter WHAT I did, it wouldn't be polished enough for me to be especially happy with a "release". This was compounded by the last week or so where I'd been beating my head on a wall trying to fix some "last bugs"(I've been doing that for a month now), and just could not, for the life of me, figure it out. I know I CAN, but it was taking far more time than it should, and I was making me hate myself over it.
That, and honestly, nothing I could charge in the Community Purchase campaign could ever REALLY make up the...what, like, 6 months of effort that's gone into this now?
So hey, whatever.
So...what, are you done with this then?
Oh God no. I plan to use this system in the games I work on, so even if no one ever touches the repo, I'll keep updating it.
This is pretty much just forcing me to let it go and get other people who are smarter than me working on it to fix crap I failed to.
The obvious endgoal is it gets polished to a mirror sheen, and then it gets rolled into the T3D development branch, and then eventually master. But it's got a lotta work to go before that's happening and I figure, why not start now.
So my repo could be considered the definitive Torque3D Entity/Component branch that we'll all(hopefully) collectively polish until we feel comfortable about rolling it into an official branch for proper testing pre-integration.
However, it's in a very, very rough state now. Tons and tons and tons of chaff from self comments, testing code and as-we-go rewrites of entire systems and implementations clog the current implementation. To say nothing of experimental behaviors I didn't get really working still sitting on my harddrive for later.
As such, I have no doubts, at all in my mind, that it'll be kinda crap to navigate or for people to really work with. But you know what? That's OK.
How is that OK? That's passing the buck, isn't it?
Shoosh, you. No, it's OK, because I'm throwing down an open invitation to talk to me here on the forums, in chat, or via email at any time for clarification, suggestions, or ideas.
This is a community project, and while I've got a pretty concrete idea of where it's going, the whole point is we'd work on it together to make it perfect.
Similarly, if any of you guys want to make your own components for stuff, or convert existing behavior in the engine/your projects into components/behaviors, feel free to ask. I'm absolutely happy to help brainstorm how to component-ize everything.
If you don't know what you want to componentize, but wanna do something, like said, I'm sitting on half-implemented stuff I think could be a pretty big deal if it was made to work right. So by all means, please fire me a line.
So it works out of the box then?
More or less, yeah.
In the branch is a new template. The EntityComponent template.
When making a new project with it, you'll have to add the missing folders/files in the engine/component directory as I haven't fixed the project template stuff yet, but after that it should compile and run fine.
The template is based on Dan Buckmaster's Torque3D Barebones template, only this has menus, editors and server/client functionality implemented as well. So it's a more "game ready" template.
Currently, it boots straight into the mission, but you can re-enable it going to the main menu by tweaking what it loads to in the main.cs.
(Again, if you have any questions, fire them at me)
So all that effort, for free?
More or less, yeah.
I mean, if you've got some extra cash and want to throw it at me? Sure, why not.
I've got a paypal, here:
www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=N3L6L64NGSDM8
Feel free to toss something my way if you think I deserve it.
With that said, grab the repo, balk at my coding atrocities, and once you get over the whiplash, lets get Torque'ing :D
-Jeff out
About the author
#22
For example, I can't think of any reason for Entities to NOT automatically be ghosted to the client and have networked support. So you wouldn't move the entire networking/ghosting system off, for example.
And yeah, I'll have to look at SceneObject for a better sense of what all it manages, but I think the biggie is it handling scene integration and transform. I hadn't looked at it not assuming for having transform(even if that transform never changes).
It definitely could be something we do, though you'd be looking at having some pretty hard dependency rules for lots of the components, since they would all assume it would have a transform. That's why I just rolled with it assuming that it would be there.
As far as I know in regards to Unity, while the transform is a component, it's part of a GameObject. I dunno if you can remove it or not(probably not though, given that in Unity, everything has to exist in the scene. Torque isn't likewise restricted)
For ticking, if we're going to look at disabling if an Entity ticks or not, it'd be a lot easier to just have a field that toggles it on or off, I think. I haven't run any testing on how much having an object that is ticked, but doesn't do anything WHILE ticked impacts performance overhead. But I think extrapolating ticking usage into a separate system would probably be overcomplicating it when we could just have a "doesTick" yes/no.
The tick overheard would be a really good thing to test though, I hadn't thought of that at all, really.
07/20/2014 (2:26 pm)
While I agree at large, another thing to consider is you hit a point where the breakdown of everything gets so extreme it starts to get detrimental.For example, I can't think of any reason for Entities to NOT automatically be ghosted to the client and have networked support. So you wouldn't move the entire networking/ghosting system off, for example.
And yeah, I'll have to look at SceneObject for a better sense of what all it manages, but I think the biggie is it handling scene integration and transform. I hadn't looked at it not assuming for having transform(even if that transform never changes).
It definitely could be something we do, though you'd be looking at having some pretty hard dependency rules for lots of the components, since they would all assume it would have a transform. That's why I just rolled with it assuming that it would be there.
As far as I know in regards to Unity, while the transform is a component, it's part of a GameObject. I dunno if you can remove it or not(probably not though, given that in Unity, everything has to exist in the scene. Torque isn't likewise restricted)
For ticking, if we're going to look at disabling if an Entity ticks or not, it'd be a lot easier to just have a field that toggles it on or off, I think. I haven't run any testing on how much having an object that is ticked, but doesn't do anything WHILE ticked impacts performance overhead. But I think extrapolating ticking usage into a separate system would probably be overcomplicating it when we could just have a "doesTick" yes/no.
The tick overheard would be a really good thing to test though, I hadn't thought of that at all, really.
#23
Then check out the small collection of links I try to curate on the topic. Some of them was mentioned in the video, some re-iterates what was said in the video, while som provide additional perspectives.
07/20/2014 (3:10 pm)
I just want to encourage you to check out the YouTube video I posted above, it covers both data-oriented design, and the issues it try to solve, entity systems, component systems and the conservative approach current engines take. One does not have to take a full head on DOD approach, it can be used in conjunction with a well-designed non-naive OOP approach.Then check out the small collection of links I try to curate on the topic. Some of them was mentioned in the video, some re-iterates what was said in the video, while som provide additional perspectives.
#24
If THAT's the case, then yeah, I can totally see us breaking down transform to a component, though we'd absolutely need to have a clean way of handling the dependencies of other components that would require of it. Rendering and physics and the like can't work if there's no transform and it's not part of the scene ;)
@Anders(you responded while I was typing this up, haha)
I'll definitely give it a look a bit later(it's fairly long ;) )
07/20/2014 (3:16 pm)
Thinking on it further, a Entity without a transform component(and thus isn't integrated into the scene) would probably basically just get used as a ScriptObject or SimObject in conventional Torque, right?If THAT's the case, then yeah, I can totally see us breaking down transform to a component, though we'd absolutely need to have a clean way of handling the dependencies of other components that would require of it. Rendering and physics and the like can't work if there's no transform and it's not part of the scene ;)
@Anders(you responded while I was typing this up, haha)
I'll definitely give it a look a bit later(it's fairly long ;) )
#26
Jeff - yeah, that'd the exact idea. Also, some of those objects don't need to be networked. Like anything to do with AI, which is usually done on the server. Heck, even per-client renderables, like 3D HUDs, waypoint markers, cloaked units, etc.
Dependencies definitely required. I'm not sure if I like the way EntityX handles this with a global manager, but I very much like the way Systems can select entities with certain components. And it's all type-safe! !!!
EDIT: Just reading through the BehaviorInterface code, which is an interesting difference - EntityX doesn't have any concept of implementing the same component in different ways (i.e. interfaces), and it seems like the guy Anders linked to doesn't like virtual methods/inheritance either. Which is interesting. We should consider carefully whether that'll work for us - could we, in fact, do away with interfaces entirely? What do we lose and gain?
07/21/2014 (1:29 am)
Anders - downloaded the vid to take a look at while travelling tomorrow, and that book looks like a treasure trove of info. Funnily enough, the idea of moving data out of classes and into maps for each 'component' is something I've seen advocated in Haskell game development. Just thought I'd mention that :P.Jeff - yeah, that'd the exact idea. Also, some of those objects don't need to be networked. Like anything to do with AI, which is usually done on the server. Heck, even per-client renderables, like 3D HUDs, waypoint markers, cloaked units, etc.
Dependencies definitely required. I'm not sure if I like the way EntityX handles this with a global manager, but I very much like the way Systems can select entities with certain components. And it's all type-safe! !!!
EDIT: Just reading through the BehaviorInterface code, which is an interesting difference - EntityX doesn't have any concept of implementing the same component in different ways (i.e. interfaces), and it seems like the guy Anders linked to doesn't like virtual methods/inheritance either. Which is interesting. We should consider carefully whether that'll work for us - could we, in fact, do away with interfaces entirely? What do we lose and gain?
#27
Hm, fair point.
When you're thinking of stuff like AI, are you referring to like, an AI Manager entity, or are you referring to an AI component? Because components can be flagged to be networked or not, it's just Entities that are always networked.
A manager of some kind for various gameplay stuff would definitely make sense ot leave on the server though.
For client-sided stuff, there's 2 things I hadn't figured out, though I'd thought about:
1) How do you create a client-only entity/component easily
2) You want to be incredibly wary of doing any rendered stuff without the server being authoritative over it in a gameplay sense(such as cloaking effects on an enemy) as it could be exploited to cheat.
What I was looking at for managing client/server visibility is exploiting ghosting info and hiding/unhiding the ghost object on the client.
It's out of date and could easily be done as a engine-level component instead, but if you look at the behaviors/render/clientVisibility.cs file, it was going to be designed so you could attach it to an entity, and dictate what clients could and could not see it.
You could easily do a masked or tagged approach as well. So for a traditional FPS, you'd have the visibility behavior on the player body entity, and one on the first-person arms and just flag visibility accordingly so the controller player sees only the first-person arms, while other players see the third person body.
I hadn't figured out any practical way to get that behavior to exist in a 100% client-side way. If you've got ideas though, I'd be all for hearing them :D
As for interfaces vs systems, I'm not sure.
The system as it is there should be pretty fast, and it gives a sane way for components to inquire after other components without actually knowing anything about the component you're querying against.
In the case of the physics-collisions components working together, we have standardized functions in the interface that pass data we can anticipate the format of. So regardless of if the physics component is a simple physics, or a rigid body, or ragdoll or whatever, and regardless of if the collisions are box, capsule, or mesh, we can ensure they work seamlessly without them directly talking.
I'm DEFINITELY not an authority on Entity/Component systems, but this struck me as being the best balance in regards to actual, practical usage. It should be pretty fast, with fairly reasonable overhead(as the interfaces are cached on the entity, but the interfaces themselves are lightweight) and would work with any components we may have.
So guess the question is would using the data-systems approach offer a sizable boost to performance or usability over that? I have no idea D:
07/21/2014 (8:05 am)
@DanHm, fair point.
When you're thinking of stuff like AI, are you referring to like, an AI Manager entity, or are you referring to an AI component? Because components can be flagged to be networked or not, it's just Entities that are always networked.
A manager of some kind for various gameplay stuff would definitely make sense ot leave on the server though.
For client-sided stuff, there's 2 things I hadn't figured out, though I'd thought about:
1) How do you create a client-only entity/component easily
2) You want to be incredibly wary of doing any rendered stuff without the server being authoritative over it in a gameplay sense(such as cloaking effects on an enemy) as it could be exploited to cheat.
What I was looking at for managing client/server visibility is exploiting ghosting info and hiding/unhiding the ghost object on the client.
It's out of date and could easily be done as a engine-level component instead, but if you look at the behaviors/render/clientVisibility.cs file, it was going to be designed so you could attach it to an entity, and dictate what clients could and could not see it.
You could easily do a masked or tagged approach as well. So for a traditional FPS, you'd have the visibility behavior on the player body entity, and one on the first-person arms and just flag visibility accordingly so the controller player sees only the first-person arms, while other players see the third person body.
I hadn't figured out any practical way to get that behavior to exist in a 100% client-side way. If you've got ideas though, I'd be all for hearing them :D
As for interfaces vs systems, I'm not sure.
The system as it is there should be pretty fast, and it gives a sane way for components to inquire after other components without actually knowing anything about the component you're querying against.
In the case of the physics-collisions components working together, we have standardized functions in the interface that pass data we can anticipate the format of. So regardless of if the physics component is a simple physics, or a rigid body, or ragdoll or whatever, and regardless of if the collisions are box, capsule, or mesh, we can ensure they work seamlessly without them directly talking.
I'm DEFINITELY not an authority on Entity/Component systems, but this struck me as being the best balance in regards to actual, practical usage. It should be pretty fast, with fairly reasonable overhead(as the interfaces are cached on the entity, but the interfaces themselves are lightweight) and would work with any components we may have.
So guess the question is would using the data-systems approach offer a sizable boost to performance or usability over that? I have no idea D:
#28
Which, to be fair, is probably our responsibility as game/engine developers. But I wonder if there's an easier way.
Note that data-oriented design doesn't necessarily entail a component/entity system, and nor does a C/E system imply that the internals have to be data-oriented. So you could theoretically use DOD principles to make the existing OO design more performant, without sacrificing its benefits. Such as they are. But I think the reason we're discussing this is because the C/E model is much nicer for making a general game engine than an OOP model with inheritance. And DOD goes hand in hand with C/E, so we're discussing that, too.
07/21/2014 (10:34 am)
This was interesting (another one for the list, Anders?). The more I read about this paradigm... well, it seems like a regression to me. We're designing code around the way the hardware works, rather than seeking higher-level abstractions.Which, to be fair, is probably our responsibility as game/engine developers. But I wonder if there's an easier way.
Note that data-oriented design doesn't necessarily entail a component/entity system, and nor does a C/E system imply that the internals have to be data-oriented. So you could theoretically use DOD principles to make the existing OO design more performant, without sacrificing its benefits. Such as they are. But I think the reason we're discussing this is because the C/E model is much nicer for making a general game engine than an OOP model with inheritance. And DOD goes hand in hand with C/E, so we're discussing that, too.
#29
Interesting article. I kinda wish he'd provided some psuedocode examples, as I prefer parsing code than reading articles where they just talk about how it COULD be, but it's interesting none the less.
As for your comment with "The more I read about this paradigm... well, it seems like a regression to me."
I'm brainfarting on the context, lol. Are you referring to the system as it is in the E/C release, or something else?
And yeah, I know it doesn't automatically imply anything, but like I said earlier, most of the development of this is was me going "I have no idea what any of you people are talking about" and then looked at actual in-use examples in actual game engines, like T2D's behaviors and Unity's components and extrapolated from there.
It's inevitably not the best implementation, but it worked and it felt more "real" to me than a foundational-level example where there's literally no code to the engine BUT the E/C system, which is definitely not how Torque currently is.
Definitely another wrinkle in writing that kinda thing :/
07/21/2014 (12:01 pm)
@Dan Interesting article. I kinda wish he'd provided some psuedocode examples, as I prefer parsing code than reading articles where they just talk about how it COULD be, but it's interesting none the less.
As for your comment with "The more I read about this paradigm... well, it seems like a regression to me."
I'm brainfarting on the context, lol. Are you referring to the system as it is in the E/C release, or something else?
And yeah, I know it doesn't automatically imply anything, but like I said earlier, most of the development of this is was me going "I have no idea what any of you people are talking about" and then looked at actual in-use examples in actual game engines, like T2D's behaviors and Unity's components and extrapolated from there.
It's inevitably not the best implementation, but it worked and it felt more "real" to me than a foundational-level example where there's literally no code to the engine BUT the E/C system, which is definitely not how Torque currently is.
Definitely another wrinkle in writing that kinda thing :/
#30
First of all, only the parts in which improved performance matters need to apply DOD principles. There are areas where they are wrong and/or hard to implement sanely (like AI behavioral trees).
Second, the important thing to recognize is that we prefer a component based architecture, that is we prefer aggregation over inheritance. That's the starting point. Exactly how it is implemented is up to us. Such an architecture can be implemented as anything from a naive OOP approach (with lots of flexibility and poor performance) to a hard-core DOD approach (with minimal flexibility but high performance) and anything in between.
Third, however if we want to implement said component architecture with reasonable good performance but still keep it flexible we need to look at what may cause any performance degradation and how to get around it, which leads us to become cache aware. This is the important bit! Not how it is implemented.
How a component based system is designed and how it is implemented are two separate things. An ES (one type of design of a components based system) can be implemented with complete lack of regard for DOD principles (altough the design is simpler to turn into a DOD implementation) while a non-ES (but still a component based system) can be the complete opposite.
One thing to look out for (which Unity is kinda bad at) is component dependencies, that to be able to process one component it must pull in alot of others to complete the task.
The problem with virtual member functions is that they cause branching. Branch prediction going wrong is one reason for cache flush and wait state. Not that they are totaly evil. (See my first point.)
Check out the blog posts Sane Usage of Components and Entity Systems and Cache Aware Components by Randy Gaul.
EDIT: Edited this a ton of times, just to make my point clear. Sorry.
07/21/2014 (1:03 pm)
@Dan: Exactly! That was kind of the point I tried to make above with "one does not have to take a full head on DOD approach".First of all, only the parts in which improved performance matters need to apply DOD principles. There are areas where they are wrong and/or hard to implement sanely (like AI behavioral trees).
Second, the important thing to recognize is that we prefer a component based architecture, that is we prefer aggregation over inheritance. That's the starting point. Exactly how it is implemented is up to us. Such an architecture can be implemented as anything from a naive OOP approach (with lots of flexibility and poor performance) to a hard-core DOD approach (with minimal flexibility but high performance) and anything in between.
Third, however if we want to implement said component architecture with reasonable good performance but still keep it flexible we need to look at what may cause any performance degradation and how to get around it, which leads us to become cache aware. This is the important bit! Not how it is implemented.
How a component based system is designed and how it is implemented are two separate things. An ES (one type of design of a components based system) can be implemented with complete lack of regard for DOD principles (altough the design is simpler to turn into a DOD implementation) while a non-ES (but still a component based system) can be the complete opposite.
One thing to look out for (which Unity is kinda bad at) is component dependencies, that to be able to process one component it must pull in alot of others to complete the task.
The problem with virtual member functions is that they cause branching. Branch prediction going wrong is one reason for cache flush and wait state. Not that they are totaly evil. (See my first point.)
Check out the blog posts Sane Usage of Components and Entity Systems and Cache Aware Components by Randy Gaul.
EDIT: Edited this a ton of times, just to make my point clear. Sorry.
#31
Anders: yeah, I do like that an ECS makes it easier to move to a DOD data layout if we want to.
So, I guess the first thing to do (once we have some stability and features!) is start benchmarking!
I'm also very sorely tempted to see if I can patch EntityX to not require variadic templates. The API would be a little less nice, but if we can bring its type-safe goodness to VS2008 (or even 2010), I think it'd be awesome to see about using something like it for our internals.
Of course, I'm still reasonably clueless as to how this will all interact with the console and networking. Sigh, silly real life getting in the way of neat abstractions.
07/21/2014 (1:46 pm)
Love the discussion going on here. Jeff, Anders basically clarified what I meant by that remark. I wasn't commenting on your work, but on the DOD paradigm as presented in the article I linked to.Anders: yeah, I do like that an ECS makes it easier to move to a DOD data layout if we want to.
So, I guess the first thing to do (once we have some stability and features!) is start benchmarking!
I'm also very sorely tempted to see if I can patch EntityX to not require variadic templates. The API would be a little less nice, but if we can bring its type-safe goodness to VS2008 (or even 2010), I think it'd be awesome to see about using something like it for our internals.
Of course, I'm still reasonably clueless as to how this will all interact with the console and networking. Sigh, silly real life getting in the way of neat abstractions.
#32
Oooh, those are some nice articles.
Reading through them, his "Sane Usage" sounds actually fairly close to where it is now, it's just leaned heavily on iterability as opposed to heavy caching for performance. I had figured that the massive boost to iteration speed and the ability to optimize the individual parts would (mostly) outweigh the inherient expensiveness otherwise.
That leads to the second post though. Part of my confusion on how to properly cache and optimize memory references for the component system arises from the fact that the components are SimObjects(and have all the associated baggage) and therefore, my understanding is, kinda harder to do super-efficient memory access optimization.
I'm not as well versed in memory management though, so I could be wrong on that front, but the SimObject nature of the components(used due to us needing console functionality for script and editor support) means they're going to be getting pinged and updated all over the place. That'd be hard for them to be used in the kind of cache approach he talks about, right?
07/21/2014 (1:53 pm)
@AndersOooh, those are some nice articles.
Reading through them, his "Sane Usage" sounds actually fairly close to where it is now, it's just leaned heavily on iterability as opposed to heavy caching for performance. I had figured that the massive boost to iteration speed and the ability to optimize the individual parts would (mostly) outweigh the inherient expensiveness otherwise.
That leads to the second post though. Part of my confusion on how to properly cache and optimize memory references for the component system arises from the fact that the components are SimObjects(and have all the associated baggage) and therefore, my understanding is, kinda harder to do super-efficient memory access optimization.
I'm not as well versed in memory management though, so I could be wrong on that front, but the SimObject nature of the components(used due to us needing console functionality for script and editor support) means they're going to be getting pinged and updated all over the place. That'd be hard for them to be used in the kind of cache approach he talks about, right?
#33
EDIT: Also, the point of all this cache awareness stuff is that you might actually be reducing performance by iterating through objects in separate cache regions, since you have a cache miss on every iteration.
Also, EntityX has a really slick API for iteration.
07/21/2014 (2:01 pm)
Oh, also Jeff, if you want an example of what DOD might look like, check out the EntityX readme. I think it's a fairly good example, though it hides the details. You can see, though, how they define little structs for each component - now imagine that all the structs of the same type are stored in a vector. So there's a vector of Position structs, a vector of Velocity structs, etc.EDIT: Also, the point of all this cache awareness stuff is that you might actually be reducing performance by iterating through objects in separate cache regions, since you have a cache miss on every iteration.
Also, EntityX has a really slick API for iteration.
#34
Yeah, definitely some good discussion!
Yeah, I think before we really do anything about deciding how to deal with the memory management, or interfaces or whatnot is to get some solid benchmark figures to find out if it's even really that bad yet.
As for EntityX, feel free to take a stab. Worse case is it just gives some ideas on how NOT to set it all up ;)
07/21/2014 (2:12 pm)
@DanYeah, definitely some good discussion!
Yeah, I think before we really do anything about deciding how to deal with the memory management, or interfaces or whatnot is to get some solid benchmark figures to find out if it's even really that bad yet.
As for EntityX, feel free to take a stab. Worse case is it just gives some ideas on how NOT to set it all up ;)
#35
Totally missed the examples in the readme for EntityX, very nice!
I do like the overall approach, though there are some pretty hefty obstacles from getting it to be that nice.
Namely, console support, necessitating SimObject, as I said.
That means not entities nor components can merely be an ID reference. They're actual objects.
So we're pretty inevitably going to be getting hit with the separate access by function and unless we ditch the console entirely(which would be a huuuge rewrite) we're not getting away from that.
Having said that, we could use some ideas and streamline things out further. For example, I ended on the interface implementation due to not being able to come up with any kind of arbitrary events system, which EntityX obviously has.
So we can look pulling parts of it as usable for sure.
Looking at how it handles systems though, it seems almost like what the interfaces are, but instead of being virtualized functions, we just process them literally, inside the interface itself.
It had occured to me, but at the time, I was looking at needing different collision functions for weither we have Box collisions, or Capsule, or Mesh or whatnot, and it basically meant we either had X number of unique interfaces, or we had the virtualized format.
Having completely unique interfaces would require a cleaner system for picking them apart(though using something like map may work for that).
I also couldn't think of a way to avoid header bloating.
For example, if we have 3 or 5 or whatever different collision interfaces for the various colliders:
Do we have them all in their own files? Do they go into a central 'collisionInterfaces' header like what I did?
In both cases it involves adding onto the headers when adding new components. If someone implemented a "coneCollider" or whatever, either it'd have it's own "coneColliderInterface" header which would need to get hooked into everything else so it can be used(again, map or something may solve that) or shoved into the core collisionInterfaces header.
So at the time, it felt like it was to...ugly? I guess? And opted for the Entity-accessed interfaces being generic and virutalized off to the owner of the interface.
Having something like what EntityX does, where it has a type map could probably solve that though. Though you'd still have to have some method to register it into the list for use.
I like the idea, but I'm not sure on the best implementation in Torque itself. Thoughts?
Edit: It also makes me wonder, if the entire point of separating out the data stored via components, and the usage of the data(actual functionality) via systems to keep everything ultra light weight - but we're stuck with full objects already by design/limitation - do we actually gain anything from separating it, instead of just having the components store data AND act on it, since it's a object already.
That'd probably be answered by benchmarking, or someone with a better grasp of the memory management stuff, but it may be a lot easier to work with like this, though at the expense of some performance.
07/21/2014 (8:40 pm)
@DanTotally missed the examples in the readme for EntityX, very nice!
I do like the overall approach, though there are some pretty hefty obstacles from getting it to be that nice.
Namely, console support, necessitating SimObject, as I said.
That means not entities nor components can merely be an ID reference. They're actual objects.
So we're pretty inevitably going to be getting hit with the separate access by function and unless we ditch the console entirely(which would be a huuuge rewrite) we're not getting away from that.
Having said that, we could use some ideas and streamline things out further. For example, I ended on the interface implementation due to not being able to come up with any kind of arbitrary events system, which EntityX obviously has.
So we can look pulling parts of it as usable for sure.
Looking at how it handles systems though, it seems almost like what the interfaces are, but instead of being virtualized functions, we just process them literally, inside the interface itself.
It had occured to me, but at the time, I was looking at needing different collision functions for weither we have Box collisions, or Capsule, or Mesh or whatnot, and it basically meant we either had X number of unique interfaces, or we had the virtualized format.
Having completely unique interfaces would require a cleaner system for picking them apart(though using something like map may work for that).
I also couldn't think of a way to avoid header bloating.
For example, if we have 3 or 5 or whatever different collision interfaces for the various colliders:
Do we have them all in their own files? Do they go into a central 'collisionInterfaces' header like what I did?
In both cases it involves adding onto the headers when adding new components. If someone implemented a "coneCollider" or whatever, either it'd have it's own "coneColliderInterface" header which would need to get hooked into everything else so it can be used(again, map or something may solve that) or shoved into the core collisionInterfaces header.
So at the time, it felt like it was to...ugly? I guess? And opted for the Entity-accessed interfaces being generic and virutalized off to the owner of the interface.
Having something like what EntityX does, where it has a type map could probably solve that though. Though you'd still have to have some method to register it into the list for use.
I like the idea, but I'm not sure on the best implementation in Torque itself. Thoughts?
Edit: It also makes me wonder, if the entire point of separating out the data stored via components, and the usage of the data(actual functionality) via systems to keep everything ultra light weight - but we're stuck with full objects already by design/limitation - do we actually gain anything from separating it, instead of just having the components store data AND act on it, since it's a object already.
That'd probably be answered by benchmarking, or someone with a better grasp of the memory management stuff, but it may be a lot easier to work with like this, though at the expense of some performance.
#36
EDIT: Whoops, I realised I can not template-ize the whole class tree if I just be a bit less dumb. Now VelocityInterface is:
EDIT: But this whole venture has been futile anyway, because of the way interfaces are used. Never mind errbody. Looks like we don't get to have elegant code...
07/23/2014 (1:27 pm)
I figured the first thing I would do is change behavior interfaces to use templates, so instead of writing this:class velInterface : public VelocityInterface
{
virtual VectorF getVelocity()
{
SimplePhysicsBehaviorInstance *bI = reinterpret_cast<SimplePhysicsBehaviorInstance*>(getOwner());
if(bI && bI->isEnabled())
return bI->getVelocity();
return VectorF(0,0,0);
}
};
velInterface mVelInterface;we could write this:VelocityInterface<SimplePhysicsBehaviorInstance> mVelInterface;because as far as I can see, all those internal class instance declarations boil down to the exact same code, with a different type that they just cast things to. But it's revealing a whole tangled web of classes around that that would all need to become templates, which is just becoming a mess.
EDIT: Whoops, I realised I can not template-ize the whole class tree if I just be a bit less dumb. Now VelocityInterface is:
template<class T>
class VelocityInterface : public BehaviorInterface
{
public:
VectorF getVelocity()
{
T* bI = static_cast<T*>(getOwner());
if(bI && bI->isEnabled())
return bI->getVelocity();
return VectorF(0,0,0);
}
};EDIT: But this whole venture has been futile anyway, because of the way interfaces are used. Never mind errbody. Looks like we don't get to have elegant code...
#37
It'd either necessitate a parent behavior type things would use(generic physics component type, for example) or stuff like the Entity class would just need to be aware of all the relevent classes to type against them, which defeats the purpose of the component stuff.
As discussed above, we can either leave the current implementation(as we're screwed when it comes to linear caching anyways, so we should only fix that if it's actually an issue), or we move functionality into the interfaces themselves and shift to a map listing for calling against them.
For now, we can probably leave it as-is, as the interface layer is small and light enough that if we have to move it to a system later, it wouldn't take much work. But yes, unfortunately no beautiful code :'(
07/23/2014 (7:32 pm)
Yeah, good idea, but that completely invalidates the idea of interfaces being generic go-betweens.It'd either necessitate a parent behavior type things would use(generic physics component type, for example) or stuff like the Entity class would just need to be aware of all the relevent classes to type against them, which defeats the purpose of the component stuff.
As discussed above, we can either leave the current implementation(as we're screwed when it comes to linear caching anyways, so we should only fix that if it's actually an issue), or we move functionality into the interfaces themselves and shift to a map listing for calling against them.
For now, we can probably leave it as-is, as the interface layer is small and light enough that if we have to move it to a system later, it wouldn't take much work. But yes, unfortunately no beautiful code :'(
#38
07/24/2014 (1:12 pm)
Is there a better place we should discuss this? I was thinking... why are interfaces separate from components? AFAICT, the purpose of an interface is to say 'find me a component on which I can call "getVelocity"'. Could components not do that themselves? Since the whole system relies on dynamic_cast anyway, that is. I guess this would look like components inheriting particular interfaces.
#39
www.garagegames.com/community/forums/viewthread/138530
Blammo.
07/24/2014 (1:57 pm)
Fair enough Dan:www.garagegames.com/community/forums/viewthread/138530
Blammo.
#40
07/25/2014 (3:57 pm)
Also, one last question(for now), is it possible to use T3D's current inheritance hell entity system alongside your ECS? 
Torque Owner Daniel Buckmaster
T3D Steering Committee
The idea being that 'SceneObject' as a class is just a way to refer to any entity that has a position component and a renderable component. In EntityX terms, it'd be a System that requires position and render data, and goes through whatever render loop the engine already does.
GameBase could be replaced by a System that requires tick reception and... I dunno, something else. Or possibly just ticks. Or possibly it doesn't matter.
You're right, EntityX is really designed to be used from C++, but I think its API is a great thing to look up to. I haven't thought in detail how anything like it might be integrated with TS or the console in general...