Entity/Component System Released!
by Jeff Raab · 07/18/2014 (6:12 pm) · 45 comments
Hey all!
github.com/Areloch/Torque3D/tree/EntityComponent_Experimental
BAM.
Wait, what? You're releasing it? With no community purchase campaign? Are you sick? Dying? IS THIS YOUR FINAL WILL AND TESTAMENT?!
What? No, that's stupid.
Whew, ok, yeah, that got a little dark there. But seriously, what's going on?
It pretty much boils down to the fact that no matter WHAT I did, it wouldn't be polished enough for me to be especially happy with a "release". This was compounded by the last week or so where I'd been beating my head on a wall trying to fix some "last bugs"(I've been doing that for a month now), and just could not, for the life of me, figure it out. I know I CAN, but it was taking far more time than it should, and I was making me hate myself over it.
That, and honestly, nothing I could charge in the Community Purchase campaign could ever REALLY make up the...what, like, 6 months of effort that's gone into this now?
So hey, whatever.
So...what, are you done with this then?
Oh God no. I plan to use this system in the games I work on, so even if no one ever touches the repo, I'll keep updating it.
This is pretty much just forcing me to let it go and get other people who are smarter than me working on it to fix crap I failed to.
The obvious endgoal is it gets polished to a mirror sheen, and then it gets rolled into the T3D development branch, and then eventually master. But it's got a lotta work to go before that's happening and I figure, why not start now.
So my repo could be considered the definitive Torque3D Entity/Component branch that we'll all(hopefully) collectively polish until we feel comfortable about rolling it into an official branch for proper testing pre-integration.
However, it's in a very, very rough state now. Tons and tons and tons of chaff from self comments, testing code and as-we-go rewrites of entire systems and implementations clog the current implementation. To say nothing of experimental behaviors I didn't get really working still sitting on my harddrive for later.
As such, I have no doubts, at all in my mind, that it'll be kinda crap to navigate or for people to really work with. But you know what? That's OK.
How is that OK? That's passing the buck, isn't it?
Shoosh, you. No, it's OK, because I'm throwing down an open invitation to talk to me here on the forums, in chat, or via email at any time for clarification, suggestions, or ideas.
This is a community project, and while I've got a pretty concrete idea of where it's going, the whole point is we'd work on it together to make it perfect.
Similarly, if any of you guys want to make your own components for stuff, or convert existing behavior in the engine/your projects into components/behaviors, feel free to ask. I'm absolutely happy to help brainstorm how to component-ize everything.
If you don't know what you want to componentize, but wanna do something, like said, I'm sitting on half-implemented stuff I think could be a pretty big deal if it was made to work right. So by all means, please fire me a line.
So it works out of the box then?
More or less, yeah.
In the branch is a new template. The EntityComponent template.
When making a new project with it, you'll have to add the missing folders/files in the engine/component directory as I haven't fixed the project template stuff yet, but after that it should compile and run fine.
The template is based on Dan Buckmaster's Torque3D Barebones template, only this has menus, editors and server/client functionality implemented as well. So it's a more "game ready" template.
Currently, it boots straight into the mission, but you can re-enable it going to the main menu by tweaking what it loads to in the main.cs.
(Again, if you have any questions, fire them at me)
So all that effort, for free?
More or less, yeah.
I mean, if you've got some extra cash and want to throw it at me? Sure, why not.
I've got a paypal, here:
www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=N3L6L64NGSDM8
Feel free to toss something my way if you think I deserve it.
With that said, grab the repo, balk at my coding atrocities, and once you get over the whiplash, lets get Torque'ing :D
-Jeff out
github.com/Areloch/Torque3D/tree/EntityComponent_Experimental
BAM.
Wait, what? You're releasing it? With no community purchase campaign? Are you sick? Dying? IS THIS YOUR FINAL WILL AND TESTAMENT?!
What? No, that's stupid.
Whew, ok, yeah, that got a little dark there. But seriously, what's going on?
It pretty much boils down to the fact that no matter WHAT I did, it wouldn't be polished enough for me to be especially happy with a "release". This was compounded by the last week or so where I'd been beating my head on a wall trying to fix some "last bugs"(I've been doing that for a month now), and just could not, for the life of me, figure it out. I know I CAN, but it was taking far more time than it should, and I was making me hate myself over it.
That, and honestly, nothing I could charge in the Community Purchase campaign could ever REALLY make up the...what, like, 6 months of effort that's gone into this now?
So hey, whatever.
So...what, are you done with this then?
Oh God no. I plan to use this system in the games I work on, so even if no one ever touches the repo, I'll keep updating it.
This is pretty much just forcing me to let it go and get other people who are smarter than me working on it to fix crap I failed to.
The obvious endgoal is it gets polished to a mirror sheen, and then it gets rolled into the T3D development branch, and then eventually master. But it's got a lotta work to go before that's happening and I figure, why not start now.
So my repo could be considered the definitive Torque3D Entity/Component branch that we'll all(hopefully) collectively polish until we feel comfortable about rolling it into an official branch for proper testing pre-integration.
However, it's in a very, very rough state now. Tons and tons and tons of chaff from self comments, testing code and as-we-go rewrites of entire systems and implementations clog the current implementation. To say nothing of experimental behaviors I didn't get really working still sitting on my harddrive for later.
As such, I have no doubts, at all in my mind, that it'll be kinda crap to navigate or for people to really work with. But you know what? That's OK.
How is that OK? That's passing the buck, isn't it?
Shoosh, you. No, it's OK, because I'm throwing down an open invitation to talk to me here on the forums, in chat, or via email at any time for clarification, suggestions, or ideas.
This is a community project, and while I've got a pretty concrete idea of where it's going, the whole point is we'd work on it together to make it perfect.
Similarly, if any of you guys want to make your own components for stuff, or convert existing behavior in the engine/your projects into components/behaviors, feel free to ask. I'm absolutely happy to help brainstorm how to component-ize everything.
If you don't know what you want to componentize, but wanna do something, like said, I'm sitting on half-implemented stuff I think could be a pretty big deal if it was made to work right. So by all means, please fire me a line.
So it works out of the box then?
More or less, yeah.
In the branch is a new template. The EntityComponent template.
When making a new project with it, you'll have to add the missing folders/files in the engine/component directory as I haven't fixed the project template stuff yet, but after that it should compile and run fine.
The template is based on Dan Buckmaster's Torque3D Barebones template, only this has menus, editors and server/client functionality implemented as well. So it's a more "game ready" template.
Currently, it boots straight into the mission, but you can re-enable it going to the main menu by tweaking what it loads to in the main.cs.
(Again, if you have any questions, fire them at me)
So all that effort, for free?
More or less, yeah.
I mean, if you've got some extra cash and want to throw it at me? Sure, why not.
I've got a paypal, here:
www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=N3L6L64NGSDM8
Feel free to toss something my way if you think I deserve it.
With that said, grab the repo, balk at my coding atrocities, and once you get over the whiplash, lets get Torque'ing :D
-Jeff out
About the author
#3
EDIT: Seriously, you're awesome for doing this Jeff, and for open-sourcing it! I'll be tossing you some pennies soon :). Can't wait to get stuck into this.
07/19/2014 (12:47 am)
EDIT: Seriously, you're awesome for doing this Jeff, and for open-sourcing it! I'll be tossing you some pennies soon :). Can't wait to get stuck into this.
#4
I probably won't use it for some time as Real life is dominating my destiny at present :-(
07/19/2014 (3:56 am)
Thrown you some pennies for this, It's not much but better than nothing, hope others follow suit.I probably won't use it for some time as Real life is dominating my destiny at present :-(
#5
Like so many other efforts (including Torque itself before opensource) there is little or no documentation.
Not being critical here but I don't know what I don't know. The GIT repo just looks like T3D - what am I missing.
Appreciate your efforts, just not sure what it is you've done....
(What am I missing?)
Thanks
07/19/2014 (4:46 am)
This could be the best mouse trap in the world, but I have no idea how to use it to catch mice.Like so many other efforts (including Torque itself before opensource) there is little or no documentation.
Not being critical here but I don't know what I don't know. The GIT repo just looks like T3D - what am I missing.
Appreciate your efforts, just not sure what it is you've done....
(What am I missing?)
Thanks
#6
that's cool.
was dreaming of this system and was waiting to see what form it has got over the last 6 month.
This system + an android /ios port can make the things happen to t3d.
Thanks Jeff for your generosity and hard work.
07/19/2014 (5:06 am)
ooh,that's cool.
was dreaming of this system and was waiting to see what form it has got over the last 6 month.
This system + an android /ios port can make the things happen to t3d.
Thanks Jeff for your generosity and hard work.
#7
07/19/2014 (5:18 am)
Thanks Jeff..Alot of us have been waiting for this so I'm sure it'll get some use!
#8
However it doesn't make sense to create documentation for WIP stuff, this has (as I understand it) not finished development.
If I'd written documentation for my Taml branch as soon as it was in a working condition I'd have had to rewrite several times already :P
@Jeff the inevitable question.. Whats the performance of this and what are the downsides?
Anyways this looks great, thanks a ton and great job I'll see if my wallet is still alive.. Maybe there are some pennies down there you can have..
Edit: Huh it's still donates money in Fuzzyvoid name.. Need to change that lol.
Another edit: I take that back, there is plenty of documentation in the code files :P
07/19/2014 (5:36 am)
@James I'd be happy to create some documentation for it as soon as it is in stock T3D.However it doesn't make sense to create documentation for WIP stuff, this has (as I understand it) not finished development.
If I'd written documentation for my Taml branch as soon as it was in a working condition I'd have had to rewrite several times already :P
@Jeff the inevitable question.. Whats the performance of this and what are the downsides?
Anyways this looks great, thanks a ton and great job I'll see if my wallet is still alive.. Maybe there are some pennies down there you can have..
Edit: Huh it's still donates money in Fuzzyvoid name.. Need to change that lol.
Another edit: I take that back, there is plenty of documentation in the code files :P
#9
Woah hey, thanks everyone :D
@James
When you pull the clone from git, make sure you swap the branch to the EntityComponent_Experimental branch.
And yeah, there's near nothing for documentation, hence all my talking in my post about it being an incomplete release.
The good news is, I fully plan to keep working on this and we'll get it sexied-up as we go. Documentation on everything is definitely something I'll be doing stuff on in the very near future.
I figured the first step was getting it out ;)
@Lukas
Not sure what the performance is in a stress-test situation. It should perform fairly well as I tried to keep the "glue" between stuff as thin as possible, but I hadn't personally done any stress testing yet.
As far as I know, the heaviest impact you'll take currently is that you're making a fair bit more simObjects for the various components.
I think you could optimize that pretty hard though, and I've got a few ideas on where we can take it if you wanted to go to the extreme(it would involve ditching TS support though D:)
07/19/2014 (6:13 am)
@AllWoah hey, thanks everyone :D
@James
When you pull the clone from git, make sure you swap the branch to the EntityComponent_Experimental branch.
And yeah, there's near nothing for documentation, hence all my talking in my post about it being an incomplete release.
The good news is, I fully plan to keep working on this and we'll get it sexied-up as we go. Documentation on everything is definitely something I'll be doing stuff on in the very near future.
I figured the first step was getting it out ;)
@Lukas
Not sure what the performance is in a stress-test situation. It should perform fairly well as I tried to keep the "glue" between stuff as thin as possible, but I hadn't personally done any stress testing yet.
As far as I know, the heaviest impact you'll take currently is that you're making a fair bit more simObjects for the various components.
I think you could optimize that pretty hard though, and I've got a few ideas on where we can take it if you wanted to go to the extreme(it would involve ditching TS support though D:)
#10
07/19/2014 (8:01 am)
@Jeff, would it be possible to make weapons with this system and would it be possible to attach sights to them?
#11
Definitively.
I was looking to do a specialized editor to make it easier to work with, but there's a StateMachineBehavior in there right now. You could manually create the state and transition rules via script to replicate the ShapeImage state machine stuff.
From there, attachments would be just mounting other entities to your weapon entity.
There's several ways you can do it, including mounting to specific nodes in a shape via the RenderShapeBehavior's mountObject() console call, or just mounting to the weapon and offsetting the position and rotation.
07/19/2014 (8:13 am)
@ChrisDefinitively.
I was looking to do a specialized editor to make it easier to work with, but there's a StateMachineBehavior in there right now. You could manually create the state and transition rules via script to replicate the ShapeImage state machine stuff.
From there, attachments would be just mounting other entities to your weapon entity.
There's several ways you can do it, including mounting to specific nodes in a shape via the RenderShapeBehavior's mountObject() console call, or just mounting to the weapon and offsetting the position and rotation.
#13
The component system merged almost flawlessly with my Taml implementation!
I was able to write the Gun object into a .taml file you can get the Taml/Component branch here, almost feels like writing T2D scripts now :P
07/19/2014 (8:39 am)
#showoffThe component system merged almost flawlessly with my Taml implementation!
I was able to write the Gun object into a .taml file you can get the Taml/Component branch here, almost feels like writing T2D scripts now :P
#14
Lukas' post also reminds me to mention:
I have set up issue tracking on my repo, so any and all issues, documentation requests, etc can be filed as an issue on the repo so we can all better track what needs to be done.
I did Lukas a favor by making an issue on my repo pointing to the TAML integration ;D
07/19/2014 (9:05 am)
ohyou.jpg ;)Lukas' post also reminds me to mention:
I have set up issue tracking on my repo, so any and all issues, documentation requests, etc can be filed as an issue on the repo so we can all better track what needs to be done.
I did Lukas a favor by making an issue on my repo pointing to the TAML integration ;D
#15
Many times, just documenting the functions and classes this way saves users tons of time reading code. The "big picture" docs can come later - the "This is the design, this is the intended use, this is the roadmap" stuff.
I am guilty of this as often as not. I frequently come back to code months later and scratch my head for a few hours because I didn't update those little bits - just a few "notes to self" scattered around like dirty socks. It always seems like nobody wants to document anything - even me, and I complain about lack of docs all the time....
07/19/2014 (10:12 am)
Thanks Jeff!Quote:Ah, yes - but if you'd pick a markup for your code comments you could make all of this much easier. It only takes a minute to update the "document fragment" that sits next to your code. The hardest thing is building this habit. Then you can use an automated tool to generate a base document set for your code.
If I'd written documentation for my Taml branch as soon as it was in a working condition I'd have had to rewrite several times already :P
Many times, just documenting the functions and classes this way saves users tons of time reading code. The "big picture" docs can come later - the "This is the design, this is the intended use, this is the roadmap" stuff.
I am guilty of this as often as not. I frequently come back to code months later and scratch my head for a few hours because I didn't update those little bits - just a few "notes to self" scattered around like dirty socks. It always seems like nobody wants to document anything - even me, and I complain about lack of docs all the time....
#16
07/19/2014 (2:18 pm)
I'm going to write a mini-tutorial in t3d-bones for this as soon as I can get to it. People need to feel the awesome. 
#17
Thx a lot Jeff, this are going to be great for T3D.
Time to add money to my paypal :D (Add a paypal button, it's more visible)
About performance:
All Entity/Component system has same problem. All data of a Entity it's spread on different memory locations. This cause performance problems.
No easy sollution for this :(
Use Component pools A recomendation i can do it's move Components to a memory pool(per Component type) and do frame updates per Component type intead of per Entity.
Preallocate memory for Entity/Components Another thing we can do (i use this on my projects) it's allow create static Entity/Component compositions. We can create c++ class Entity with Components as members or preallocate a block of memory for Entity and Components, for have memory packed and contiguous.
07/20/2014 (2:59 am)
Thx a lot Jeff, this are going to be great for T3D.
Time to add money to my paypal :D (Add a paypal button, it's more visible)
About performance:
All Entity/Component system has same problem. All data of a Entity it's spread on different memory locations. This cause performance problems.
No easy sollution for this :(
#18
And to add to Luis post above, e.g. this article discuss performance and memory structures for ES:
t-machine.org/index.php/2014/03/08/data-structures-for-entity-systems-contiguous...
And a nice talk about it:
www.youtube.com/watch?v=16ZF9XqkfRY
While the processing power has increased over the last 20 years the memory bus speed has not increased proportionally, so the rule of thumb today is to do more processing on the CPU/GPU from a smaller data set rather than transfer pre-calculated data.
Also, anyone interested in entity systems and data-oriented design should check out this ebook project:
www.dataorienteddesign.com/dodmain/dodmain.html
07/20/2014 (10:44 am)
That's fantastic! I just checked the blogs on a whim and was delighted to see this. Bravo.And to add to Luis post above, e.g. this article discuss performance and memory structures for ES:
t-machine.org/index.php/2014/03/08/data-structures-for-entity-systems-contiguous...
And a nice talk about it:
www.youtube.com/watch?v=16ZF9XqkfRY
While the processing power has increased over the last 20 years the memory bus speed has not increased proportionally, so the rule of thumb today is to do more processing on the CPU/GPU from a smaller data set rather than transfer pre-calculated data.
Also, anyone interested in entity systems and data-oriented design should check out this ebook project:
www.dataorienteddesign.com/dodmain/dodmain.html
#19
I really like the way EntityX says 'Components are just data' and puts all the behavior in Systems.
Shame EntityX has fairly demanding compiler requirements :/. It relies on variadic templates, which means it's got a really, really nice typesafe API. But also that it requires VS2013 :P.
EDIT: And thanks for the links, Anders!
Unfortunately I think we have lots of work to do before we even think about this more advanced stuff. The existing components need to be stable enough to replace large parts of the ShapeBase family tree. I haven't played with the code much, but the first thing I tried to do caused a crash ;P.
07/20/2014 (12:16 pm)
Luis: I like the way you're thinking. Check this out. It seems to solve a lot of the problems you're talking about. Also, Systems are a great idea for reducing the last vestiges of class hierarchy that remain in Jeff's work - for example, Entity still is basically a pared-down SceneObject with position information and so on.I really like the way EntityX says 'Components are just data' and puts all the behavior in Systems.
Shame EntityX has fairly demanding compiler requirements :/. It relies on variadic templates, which means it's got a really, really nice typesafe API. But also that it requires VS2013 :P.
EDIT: And thanks for the links, Anders!
Unfortunately I think we have lots of work to do before we even think about this more advanced stuff. The existing components need to be stable enough to replace large parts of the ShapeBase family tree. I haven't played with the code much, but the first thing I tried to do caused a crash ;P.
#20
Admittedly, I wasn't fond of the Component-System approach, where components are purely data and Systems actually DO things, if only because that makes a absolute mess of trying to do anything in a script language.
We'll definitely be able to optimize it as we test it more, however unless we figure out some workaround method, as long as components are SimObjects, you're going to have inefficiencies in having large numbers of SimObjects. However, it being a SimObject is necessary for script support - which is necessary for editor integration.
Definitely good stuff to start contemplating early on though, so those links are awesome.
@Dan, yeah, we definitely need to push to get the current functionality of the other game objects into component form first. I know Lukas was already working on particle emitters and the like. But being able to snip out shapebase(and all it's derived classes) would be a massive boost.
Also, when you say "Also, Systems are a great idea for reducing the last vestiges of class hierarchy that remain in Jeff's work - for example, Entity still is basically a pared-down SceneObject with position information and so on." are you meaning having Entity replace SceneObject?
I'd given very brief mullings on that, and it'd probably be good to do, though I don't know off the top of my head if that'd gain us too much other than streamlining the line of inheritance even more. I'll have to look into what all SceneObject does and how it impacts Entity. At minimum, GameBase down needs to get purged ;)
07/20/2014 (1:16 pm)
Yeah, quite a bit of that sort of stuff I'd looked into for what was "best", but with all the different schools of thought on it, it was kinda hard to decide. In the end I pretty much just operated on practical examples I could find, like other engines that had an E/C system implemented - at least I knew that worked "well enough".Admittedly, I wasn't fond of the Component-System approach, where components are purely data and Systems actually DO things, if only because that makes a absolute mess of trying to do anything in a script language.
We'll definitely be able to optimize it as we test it more, however unless we figure out some workaround method, as long as components are SimObjects, you're going to have inefficiencies in having large numbers of SimObjects. However, it being a SimObject is necessary for script support - which is necessary for editor integration.
Definitely good stuff to start contemplating early on though, so those links are awesome.
@Dan, yeah, we definitely need to push to get the current functionality of the other game objects into component form first. I know Lukas was already working on particle emitters and the like. But being able to snip out shapebase(and all it's derived classes) would be a massive boost.
Also, when you say "Also, Systems are a great idea for reducing the last vestiges of class hierarchy that remain in Jeff's work - for example, Entity still is basically a pared-down SceneObject with position information and so on." are you meaning having Entity replace SceneObject?
I'd given very brief mullings on that, and it'd probably be good to do, though I don't know off the top of my head if that'd gain us too much other than streamlining the line of inheritance even more. I'll have to look into what all SceneObject does and how it impacts Entity. At minimum, GameBase down needs to get purged ;)
Chris DeBoy