New TGE Branch (2.0?)
by Prairie Games · in Torque Game Engine · 03/02/2003 (10:45 am) · 74 replies
Isn't it about time to branch the TGE code? perhaps towards TGE 2.0?
It would be great to address some of the more fused architecture... *cough* 138k player.cc *cough* without the worries of breaking current games in production.
Torque would really benefit from better interfaces and code seperation... obviously, this is a pretty large task, a branch would at least allow it to be *started*...
-Joshua Ritter
ActionRPG Revolutionary
It would be great to address some of the more fused architecture... *cough* 138k player.cc *cough* without the worries of breaking current games in production.
Torque would really benefit from better interfaces and code seperation... obviously, this is a pretty large task, a branch would at least allow it to be *started*...
-Joshua Ritter
ActionRPG Revolutionary
#62
03/05/2003 (10:09 am)
Speedy space invaders is still just speedy space invaders.
#63
If anything, I'd say some attempts at modularity in the _current_ code base aren't very good for the Engine's speed.
Take the current rendering setup. The business with canonical state and self-rendering objects is nice and modular, but it's terrible if you want to minimize state switching and batch as much geometry as possible, which is the key to efficiency on T&L cards. The primary goal for a rendering system _ought_ to be to get the heck out of the video card's way, not to wrap objects up in layers of modularity so they won't interfere with each other.
03/05/2003 (10:12 am)
I'd much rather have an engine that runs fast today than one that will run fast 5 years from now.If anything, I'd say some attempts at modularity in the _current_ code base aren't very good for the Engine's speed.
Take the current rendering setup. The business with canonical state and self-rendering objects is nice and modular, but it's terrible if you want to minimize state switching and batch as much geometry as possible, which is the key to efficiency on T&L cards. The primary goal for a rendering system _ought_ to be to get the heck out of the video card's way, not to wrap objects up in layers of modularity so they won't interfere with each other.
#64
1) The development of torque being "too much work" for GG guys. This doesnt really fit, because they can simply farm out some of the work. Effectively theyve got a pretty vast range of fairly good resources code wise to draw on. No need for open source.
2) Keeping another branch so that "user modifications" can be made and everyone can get them.
Ok, so what if I DONT want billy-blogg's new fxNuclearAttack objects in my codebase? at some point, its going to come down to someone to decide what goes in and what doesnt. If you simply shove everything contributed in there, it'll go badly, if you dont, people will gripe about it not being included.
The issue here is really that there isnt enough time for the gg guys to approve and activate all the changes personally. Much less actually check them for quality and poll to see if change X should actually be incorporated.
Frankly, I dont see this second issue as a problem. If I want to incorporate a feature, I can. If not, I dont. For bug fixes and such, it would be simple for gg to enable some others to check and incorporate specific fixes (which is whats been happening).
So the only issue is, when do you accept a feature freeze from the GG guys and start working towards your own code base.
For instance, if I were doing this fulltime, I'd probably have scrapped a lot of the code and restructured it. As Brain mentioned, fairly sweeping changes. Thats of course *IF* I had both the time and it WASNT viable for me to create my game with whats there.
Fact is, whats there works, make the best of it, let the codebase develop and use it as/when you need it and have it available.
Currently, I really dont see whats wrong with the process as it is now.
Phil.
03/05/2003 (10:24 am)
There are two points I dont see being too much of a problem:1) The development of torque being "too much work" for GG guys. This doesnt really fit, because they can simply farm out some of the work. Effectively theyve got a pretty vast range of fairly good resources code wise to draw on. No need for open source.
2) Keeping another branch so that "user modifications" can be made and everyone can get them.
Ok, so what if I DONT want billy-blogg's new fxNuclearAttack objects in my codebase? at some point, its going to come down to someone to decide what goes in and what doesnt. If you simply shove everything contributed in there, it'll go badly, if you dont, people will gripe about it not being included.
The issue here is really that there isnt enough time for the gg guys to approve and activate all the changes personally. Much less actually check them for quality and poll to see if change X should actually be incorporated.
Frankly, I dont see this second issue as a problem. If I want to incorporate a feature, I can. If not, I dont. For bug fixes and such, it would be simple for gg to enable some others to check and incorporate specific fixes (which is whats been happening).
So the only issue is, when do you accept a feature freeze from the GG guys and start working towards your own code base.
For instance, if I were doing this fulltime, I'd probably have scrapped a lot of the code and restructured it. As Brain mentioned, fairly sweeping changes. Thats of course *IF* I had both the time and it WASNT viable for me to create my game with whats there.
Fact is, whats there works, make the best of it, let the codebase develop and use it as/when you need it and have it available.
Currently, I really dont see whats wrong with the process as it is now.
Phil.
#65
When I talk about wanting to see Torque get modularized, I envision a scenario wherein all of the interdependancies are broken to a point that any module with a given functionality will work.
It's the "most marvelous whizbang sound engine ever devised by mortal man" example as outlined by Rodney. The ability to simply pull and replace "chunks" of the total engine.
The fact is, that as it stands now, Torque appears to be modular (at least in my workspace it does ;-) but really isn't. When you start trudging through the code you quickly find that everything is pretty well intertwined.
I sometimes feel like I am unravelling an endless length of tangled cords... but then that may just be me.
As for the Networking code example you mentioned Davis... I would imagine that anything pertaining to the networking code should be closely guarded. Would changes to this effect evrything else in the engine? Of course they would... that code pretty well constitutes a healthy portion of the CORE of the engine, because in some way, everything else relies upon it.
Modularizing everything would actually make things a great deal easier in my opinion. Even when/ if a networking change got rolled out, it would be alot easier to update several different MODULES independantly to reflect those changes, rather than chasing down all relevant references throughout the ENTIRE engine.
I'm no expert, not by a long shot... remember C programmer! ;-) Don't get me wrong, I like Torque,. and it is hands down the best engine I've ever used, but seeing that code for the first time was quite an eye opener. It was intimidating and tedious and still is.
As for the speed issue... Torque is allready showing it's age. Heck, I can run Tribes 2 with consistently over 100 fps at any given time, with any given number of players. (Save of course for latency in the connection.) That said... while it's nice to see 160+fps in my project, it is not necessary, and makes no discernable difference. Maybe it's just me, but anything past 70 is gravy.
Would I sacrifice a few fps to gain some ease of use? YES! We're not talking about any changes that would require 5 years of hardware development to catch up with! Cripes... this isn't Doom 3 ppl!
And as a parting thought, before I turn the floor back over to the true experts, we ARE talking about a BRANCH. If the modularized version of Torque, didn't appear to suit your needs, just go back to the other branch and everyone is happy.(?)
With the breadth of talent available within this community, I'm sure that 2 'camps' could easily be supported, but then that opens a whole 'nother can 'o worms, so maybe I ought to just quit while I'm ahead eh?! LOL
*Just My 2 Cents*
03/05/2003 (10:44 am)
I think we probably need some clarity here when we talk about modularity.When I talk about wanting to see Torque get modularized, I envision a scenario wherein all of the interdependancies are broken to a point that any module with a given functionality will work.
It's the "most marvelous whizbang sound engine ever devised by mortal man" example as outlined by Rodney. The ability to simply pull and replace "chunks" of the total engine.
The fact is, that as it stands now, Torque appears to be modular (at least in my workspace it does ;-) but really isn't. When you start trudging through the code you quickly find that everything is pretty well intertwined.
I sometimes feel like I am unravelling an endless length of tangled cords... but then that may just be me.
As for the Networking code example you mentioned Davis... I would imagine that anything pertaining to the networking code should be closely guarded. Would changes to this effect evrything else in the engine? Of course they would... that code pretty well constitutes a healthy portion of the CORE of the engine, because in some way, everything else relies upon it.
Modularizing everything would actually make things a great deal easier in my opinion. Even when/ if a networking change got rolled out, it would be alot easier to update several different MODULES independantly to reflect those changes, rather than chasing down all relevant references throughout the ENTIRE engine.
I'm no expert, not by a long shot... remember C programmer! ;-) Don't get me wrong, I like Torque,. and it is hands down the best engine I've ever used, but seeing that code for the first time was quite an eye opener. It was intimidating and tedious and still is.
As for the speed issue... Torque is allready showing it's age. Heck, I can run Tribes 2 with consistently over 100 fps at any given time, with any given number of players. (Save of course for latency in the connection.) That said... while it's nice to see 160+fps in my project, it is not necessary, and makes no discernable difference. Maybe it's just me, but anything past 70 is gravy.
Would I sacrifice a few fps to gain some ease of use? YES! We're not talking about any changes that would require 5 years of hardware development to catch up with! Cripes... this isn't Doom 3 ppl!
And as a parting thought, before I turn the floor back over to the true experts, we ARE talking about a BRANCH. If the modularized version of Torque, didn't appear to suit your needs, just go back to the other branch and everyone is happy.(?)
With the breadth of talent available within this community, I'm sure that 2 'camps' could easily be supported, but then that opens a whole 'nother can 'o worms, so maybe I ought to just quit while I'm ahead eh?! LOL
*Just My 2 Cents*
#66
Issue #1: GarageGames and Torque are about the community as much as they are anything. So we have this great community of people sharing and contributing. But since virtualy none of the community changes make it into the "real" tree you've got a community of forks that make staying up to date a nightmare. There's no clear path for getting things into the main tree. Now GG can just say "No non-GG staff code in the main tree" but to turn down contributions from within their own community on a wholesale basis would be fool hardy to say the least.
I don't think anyone is saying it's too hard for them only that they're stretched VERY thin. Forget that open source was even mentioned. It was simply an idea to help advance the engine not a demand.
And of COURSE there will be gatekeepers who decide what goes in and what doesn't. That's why you don't give commit rights to everyone and you establish clear submission guidelines. Things of a generic nature that are of clear benefit to multiple projects get in, "Joe Bobs bovine launching sling shot" doesn't. This is the HEART of open/shared source development.
Issue #2: The current design precludes easy modification and customization. One has to look no further than the TerrainManager. Because of the coupling within the engine design TerrainManager has tendrils that reach virtualy every segment of the engine. Wouldn't it be nice to "mix and match", or heck even to be able to drop portions you didn't need without breaking seemingly unrelated things. TGE wasn't created to be a general purpose engine, it was built to be Tribes 2 plain and simple. I'm not, nor is anyone else, trashing TGE it's great stuff, it's definitly the path of least resistance for indies. It just needs to grow and evolve a little.
03/05/2003 (11:48 am)
@Phil I think you're missing the point on some of this...Issue #1: GarageGames and Torque are about the community as much as they are anything. So we have this great community of people sharing and contributing. But since virtualy none of the community changes make it into the "real" tree you've got a community of forks that make staying up to date a nightmare. There's no clear path for getting things into the main tree. Now GG can just say "No non-GG staff code in the main tree" but to turn down contributions from within their own community on a wholesale basis would be fool hardy to say the least.
I don't think anyone is saying it's too hard for them only that they're stretched VERY thin. Forget that open source was even mentioned. It was simply an idea to help advance the engine not a demand.
And of COURSE there will be gatekeepers who decide what goes in and what doesn't. That's why you don't give commit rights to everyone and you establish clear submission guidelines. Things of a generic nature that are of clear benefit to multiple projects get in, "Joe Bobs bovine launching sling shot" doesn't. This is the HEART of open/shared source development.
Issue #2: The current design precludes easy modification and customization. One has to look no further than the TerrainManager. Because of the coupling within the engine design TerrainManager has tendrils that reach virtualy every segment of the engine. Wouldn't it be nice to "mix and match", or heck even to be able to drop portions you didn't need without breaking seemingly unrelated things. TGE wasn't created to be a general purpose engine, it was built to be Tribes 2 plain and simple. I'm not, nor is anyone else, trashing TGE it's great stuff, it's definitly the path of least resistance for indies. It just needs to grow and evolve a little.
#67
I think your missing the boat entirely when you say you consistantly get over 100 fps in Tribes 2. I have a pretty decent system, anything over 80 in Tribes 2 is a rare thing, granted you may have a better system.
The point is though, you have a 'good' system, some others might not. You seem to only look at this picture from a perspective where everybody has top or recently top of the line hardware, and that isn't the case, not by a long shot.
Sacrificing 10 fps on my machine for modularity could be sacrificing 15, 20, or even 30 or more fps's on older hardware, now that is unacceptable.
As for the kind of modularity I would want put into the engine, I guess it would be more of a code cleanup instead. You have to have 'hooks' in some places, there is just no way of getting around that. The kind of modularity I'm against putting in the engine would be along the lines of having a plugin system for changing the scripting language, or an interface for changing the networkcode to something better. Why? Because these things are essential to the engine, a 'major' component of it in fact. If someone wants to change a 'major' component of the engine, then let them, but don't design the engine around the fact that somebody 'might' want to change those systems. To me, that is just bad design.
03/05/2003 (12:41 pm)
Kirby Webber:I think your missing the boat entirely when you say you consistantly get over 100 fps in Tribes 2. I have a pretty decent system, anything over 80 in Tribes 2 is a rare thing, granted you may have a better system.
The point is though, you have a 'good' system, some others might not. You seem to only look at this picture from a perspective where everybody has top or recently top of the line hardware, and that isn't the case, not by a long shot.
Sacrificing 10 fps on my machine for modularity could be sacrificing 15, 20, or even 30 or more fps's on older hardware, now that is unacceptable.
As for the kind of modularity I would want put into the engine, I guess it would be more of a code cleanup instead. You have to have 'hooks' in some places, there is just no way of getting around that. The kind of modularity I'm against putting in the engine would be along the lines of having a plugin system for changing the scripting language, or an interface for changing the networkcode to something better. Why? Because these things are essential to the engine, a 'major' component of it in fact. If someone wants to change a 'major' component of the engine, then let them, but don't design the engine around the fact that somebody 'might' want to change those systems. To me, that is just bad design.
#68
If you want things like a new terrain engine, a new networking model, what have you, you'll snag the Torque 2.0 when it comes out. If what you want is the old style engine, no problem, grab the 1.1.2 mantanace branch. If you need to modular approach, grab the new stuff.
It's about like dealing with Linux branches - 2.2 is no longer current. However, there's still a maintance group for it, fixing little bugs that showed up, and reintegrating it into what are essentially new versions of 2.2 - and releasing the new maint. branch for it after it's been tested again. (Actually, IIRC, 2.2 no longer has a maint. branch, but, it did at one point. If I'm wrong, I know someone will correct me ;-)
Don't like the new VM in 2.4, or some of the other changes that went into it? No problem, use 2.2's maint branch.
Same goes here.
Under my thought, community enhancements probably wouldn't go directly into either branch unless one of the maintainers deemed it really really nessisary (IE - more and more stuff is starting to be built off of that enhancement, etc.) There will probably never be a situation where everyone in the community can step up and check in an item as a piece of the engine - there's got to be someone to be a guard for it, otherwise you'll end up with one really unstable engine. When it comes to most enhancements, I'm with Phil - the current system isn't broke. When it comes to enhancements that provide really good value or other things are starting to be built off of 'em, those go in the engine. Bug fixes go in RFN! and get tested to make sure all is well.
Anyway... just my thought.
03/05/2003 (12:56 pm)
Robert: That's why in my proposed 'solution' above, I mentioned that Job 1 is integrating all fixes, etc., into a stable branch, then creating a maintance branch for it. In the maintance branch, no one would be modularize anything - it's just a very very stable version of what is the current HEAD right now.If you want things like a new terrain engine, a new networking model, what have you, you'll snag the Torque 2.0 when it comes out. If what you want is the old style engine, no problem, grab the 1.1.2 mantanace branch. If you need to modular approach, grab the new stuff.
It's about like dealing with Linux branches - 2.2 is no longer current. However, there's still a maintance group for it, fixing little bugs that showed up, and reintegrating it into what are essentially new versions of 2.2 - and releasing the new maint. branch for it after it's been tested again. (Actually, IIRC, 2.2 no longer has a maint. branch, but, it did at one point. If I'm wrong, I know someone will correct me ;-)
Don't like the new VM in 2.4, or some of the other changes that went into it? No problem, use 2.2's maint branch.
Same goes here.
Under my thought, community enhancements probably wouldn't go directly into either branch unless one of the maintainers deemed it really really nessisary (IE - more and more stuff is starting to be built off of that enhancement, etc.) There will probably never be a situation where everyone in the community can step up and check in an item as a piece of the engine - there's got to be someone to be a guard for it, otherwise you'll end up with one really unstable engine. When it comes to most enhancements, I'm with Phil - the current system isn't broke. When it comes to enhancements that provide really good value or other things are starting to be built off of 'em, those go in the engine. Bug fixes go in RFN! and get tested to make sure all is well.
Anyway... just my thought.
#69
I disagree. The submitting patches document has been available on the site for some time now. I helped write it. All of my original unix work was patches that I submitted to Tim that were promptly checked in to the head.
That being said, the community has gotten bigger since then and its needs have also expanded. One part of it wants a stable tree to build their games from, the other part wants new features and modularization. GG staff members have less time to deal with patches. Many of the existing associates (include myself), are involved with other projects and are not very focused on engine improvements. Thus, it may be harder to get a patch in now than it was in the past.
The development branch could solve many of these problems. More people would have write access, so that more patches are integrated. The maintainers could be community members with a greater interest in engine development than in making games. They would also keep the dev tree relatively clean, so that poor quality code or specialized features do not creep in.
Finally, the head would be kept stable, except for periodic merges with the dev tree to integrate features.
Of course, GG would incur additional expenses to set this up (bandwidth and cvs maintenance to start with). It may not be a financially viable option at this time.
03/05/2003 (12:57 pm)
Quote:There's no clear path for getting things into the main tree.
I disagree. The submitting patches document has been available on the site for some time now. I helped write it. All of my original unix work was patches that I submitted to Tim that were promptly checked in to the head.
That being said, the community has gotten bigger since then and its needs have also expanded. One part of it wants a stable tree to build their games from, the other part wants new features and modularization. GG staff members have less time to deal with patches. Many of the existing associates (include myself), are involved with other projects and are not very focused on engine improvements. Thus, it may be harder to get a patch in now than it was in the past.
The development branch could solve many of these problems. More people would have write access, so that more patches are integrated. The maintainers could be community members with a greater interest in engine development than in making games. They would also keep the dev tree relatively clean, so that poor quality code or specialized features do not creep in.
Finally, the head would be kept stable, except for periodic merges with the dev tree to integrate features.
Of course, GG would incur additional expenses to set this up (bandwidth and cvs maintenance to start with). It may not be a financially viable option at this time.
#70
The thing that bugs me is that the people responsible for integrating the patches havn't been, or are really really slow to do so. I have noticed that some things tend to get into the CVS a lot faster than others however, so at least there is still sign of life from you guys ;) Personally I wouldn't mind being apart of this 'development' branch you mention, wherein the cleanup and integrity of the engine itself is the sole purpose, I have much less use for developing games, so this would suit me perfect.
I guess we shall see what happens after GDC.
03/05/2003 (1:17 pm)
John Quigley,The thing that bugs me is that the people responsible for integrating the patches havn't been, or are really really slow to do so. I have noticed that some things tend to get into the CVS a lot faster than others however, so at least there is still sign of life from you guys ;) Personally I wouldn't mind being apart of this 'development' branch you mention, wherein the cleanup and integrity of the engine itself is the sole purpose, I have much less use for developing games, so this would suit me perfect.
I guess we shall see what happens after GDC.
#71
Also see: http://www.garagegames.com/index.php?sec=mg&mod=forums&page=result.thread&qt=966...
Even if Torque's code is proprietary, it doesn't mean it can't benefit off the hard work of selfless coders across the world...
Light a candle... and someone, get me a tissue... *sniffle*
-J
03/05/2003 (1:56 pm)
Ignoring Open Source code with liberal licenses is pretty silly... Besides, torque already uses Open Source code... Let's code our own jpeg and zlib cause we can? I don't think so...Also see: http://www.garagegames.com/index.php?sec=mg&mod=forums&page=result.thread&qt=966...
Even if Torque's code is proprietary, it doesn't mean it can't benefit off the hard work of selfless coders across the world...
Light a candle... and someone, get me a tissue... *sniffle*
-J
#73
Example: I replaced the embedded memory manager in Genesis3D with a modular, plugable memory manager of the style I have written about previously. The engine actually ran faster. This was due to a better designed memory manager, of course, but the modular nature obviously had no adverse effect in terms of overhead. I also must emphesize that this was a memory manager, a component that touches almost everything else in the engine.
Worries about modular overhead are unfounded. Any component in the engine, including the lowest level ones, can be modular with no concerns as long as the designer knows what they are doing.
[Added]
Also, it is important to always design with the future in mind. To assume that certain things will never change is a big mistake made everyday in all facets of software development. No part of any significant software system will remain unchanged over time. This is particularly true of something so tied to technology progress as a game engine.
03/05/2003 (3:10 pm)
I don't think anyone needs to worry about modular design slowing anything down. There is no decernable overhead introduced with proper modular design. Example: I replaced the embedded memory manager in Genesis3D with a modular, plugable memory manager of the style I have written about previously. The engine actually ran faster. This was due to a better designed memory manager, of course, but the modular nature obviously had no adverse effect in terms of overhead. I also must emphesize that this was a memory manager, a component that touches almost everything else in the engine.
Worries about modular overhead are unfounded. Any component in the engine, including the lowest level ones, can be modular with no concerns as long as the designer knows what they are doing.
[Added]
Also, it is important to always design with the future in mind. To assume that certain things will never change is a big mistake made everyday in all facets of software development. No part of any significant software system will remain unchanged over time. This is particularly true of something so tied to technology progress as a game engine.
#74
03/05/2003 (3:18 pm)
Complaints about modular designs slowing things down are about the same as the complaints about C++ slowing things down. People either 1) Compare crappy code to well written code. 2) Have never bothered to quantify things. 3) Heard it from someone who heard it from someone he thinks tried it once. 
Torque Owner Roger Smith