Pros and Cons of the proposed T3D and T2D Merge

#1 Let me start by saying my comments are going to be expressed from my own point of view as an individual, just my opinion. Not representing GG, either steering committee, or any other individual. I don't think there should be a full merge. A full merge is not a good idea in my mind. I'm talking about merging both engines, community, and committees entirely. I just don't see the logistics working out. T2D would be crippled and T3D would destabilize. It would take an unknown amount of time and work force to pull it off correctly, not to mention requiring multiple programmers with solid C++, Objective-C, C, and TS knowledge and expert experience with both of the engines in their current form. T2D MIT would continue on its own and T3D would become a mess of incomplete features.

With that out of the way, I think there is a better proposal. Both engines are MIT. Both engines have stuff the other could use. Both have users who know more about one engine than the other. My proposal is to share select features, but not merge completely.

For example, Torque 3D would benefit greatly from grabbing the TAML, Module, and Asset systems. Torque 2D could really use the console enhancements that were made to T3D. I would suggest we all start with those systems. Smaller scope, great benefits, less developers required. It doesn't negate requiring C++ and Torque knowledge, but the requirements drop dramatically.

Shall I keep going?

#2 Might as well be thorough Mich, you've been down that route what, twice now?

#3 @Michael I tried searching for TAML, but couldn't find out what it is, also what are the Module and Asset systems in T2D? Is Module like an entity-component system?

#4 Well, this is my suggested order based on priority and difficulty:

1. Port googletest framework
2. Write several tests to validate functionality and stability
3. Port TAML.
4. Update all scripts and "assets" in the current T3D examples to use new persistence system
5. Port Module and Asset
6. Update all scripts and old assets to be broken up into modules
7. Port all "old assets" to new asset format where necessary

Each step is going to break the engine, so it would be wise to port a system, fix bugs, update examples, fix new bugs, document the shit out of it, and then move to the next step. Probably a good idea to create a process of development/release.

I suggest the googletest framework first, because unit tests are going to be just as important as human driven tests. Write a test for every single core feature, especially anything exposed to script. This would probably take a couple of weeks with maybe 2 or 3 developers working part time on nothing else but this.

TAML is next, because the changes this port will make start at the very top of the hierarchy: ConsoleObject. If you open up Torque 2D's solution/project, you can see how it trickles down. Always remember that TAML is not a format. It's a persistence system. Serialization formats include XML, JSON, and binary. Once the source code is ported, any console API will have to be updated to T3D's. ConsoleMethod becomes DefineEngineMethod, etc.

Once objects are properly serializing and de-serializing, the next major challenge will be updating all of the editors to use TAML instead of writing out TorqueScript. Level Editor, GUI Editor, and Prefab are the first that come to mind.

All told, I'm guessing it would probably take a couple of weeks with maybe 2 or 3 developers working part time on nothing else but this.

So right there, some highly technical work, lots of C++, tons of script/editor changes, and about 4 - 6 weeks of work just for a single system. To top that off, TAML will be the easiest system to port. Let that sink in.

Ok, now before jumping to the next system or topic, it's a good idea to just focus on TAML for now. Ask questions, debate processes, throw out pros and cons.

#5 @Chris - Oh! Thanks for the reminder. Required Reading:

TAML
Asset Manager
Module Manager

#6 @ Michael. I will attempt to briefly list the thoughts and questions ...
1) How will be resolved differences misrepresented that T3D and T2D only work on the PC and on different platforms? whether there will be suffering from such, in part T2D? or maybe finish first for T3D multi platform?
2) if you ignore point 1) at the time that it may drag on, debugging and correction of all roughnesses, adding to them the gruel from mixing engines? instead of having to be polished up to shine, each individually, and then without a lot of dirt, concatenate into a single clean system?
3) I'm sure the union will be a great benefit, but in the condition that there is now, it can be shot in the head, a lot of effort and cost per connection details that not ready, I think only complicate the work of the committee, to a level "job is not feasible" .
4) short of (TAML Asset Manager Module Manager) it should be done and live!

#7 What about the order of thing for T3D? I'm all for unifying the codebase of both but honestly not for sure the order of operation but the ports of DX9/11 and open GL 3+ should be complete at least on the graphic side of things. Not for sure how Jeff's component system is going. I recall him saying something about a merge between both of them and maybe what he has in mind would work..
That way it could be in a 4.0 and maybe everything would be done..Just an idea!!

#8 Might as well get the notion out of the way: How much of a pain in the rear would we be talking about refactor-wise were we to go the route of having T2D and T3D share a core static dll to allow for organically bleeding the two resulting products into one another over time?

#9 @Azaezel my thought as well.

To me it would seem like the "best" thing would be to make T2D and T3D use the same code where applicable (I've been working on porting TAML to T3D), I've found that there are a lot of places where the code is almost the same, they have just deviated from each other so T2D has some improvements that T3D does not and vice versa. That code, and other code like that could be isolated to a folder/library and share that code between T3D and T2D. Like making a Torque API which the flavors of the engine build upon.

I wouldn't like a full merge, the complexity needed to run a 3D engine would just serve to corrupt the 2D engine, and the 3D engine only benefits marginally from having 2D capability, it just ends up being more bloat in the engine.

#10 For a start, we'd need to port T3D over to 4-spaced code :P.

#11 @Olexiy - I think I understand what you are saying, but forgive me if I misinterpret what you are either asking or commenting on:

1. T3D will be seen as a PC only engine until the master branch has more than one platform. Torque 2D natively supports Windows, OS X, iOS, Android, Linux, and Web. The T2D group is trying to spread the words about the new platform support. Merging Torque 3D and Torque 2D entirely would result in all of those lovely platforms being lost. This is one of the primary reasons why I'm against a total merge.

2. You, others, and I are kind of suggesting similar ideas. Don't try a full merge of both engines. Just share different core, common parts until they are at least closer together where a full merge is more viable in the future.

3. Exactly. The idea of creating a single engine by merging T3D and T2D is basically not feasible. It won't happen.

4. Not sure what you meant here. I don't know how to respond.

#12 @Kory - Well, I'm just focusing on the proposal of merging the two engines. I know there are several other efforts related to platforms, rendering, and usability for T3D. If one or more people are looking to merge the engines, the priorities and order I listed are focused on that topic alone. I'm not speaking for the steering committees. If there are higher priorities than porting T2D systems, I won't argue.

#13

Quote:How much of a pain in the rear would we be talking about refactor-wise were we to go the route of having T2D and T3D share a core static dll to allow for organically bleeding the two resulting products into one another over time?

Without a more explanation of this suggestion and a technical analysis, the cautious side of me will respond with "Severe rectal bleeding" in regard to the level of "pain in the rear." This measurement will change if you go into further details.

#14 @Lukas - I think you and the others are getting it. Stop thinking in terms of "merge the engines." That's just a pie in the sky proposal that is doomed to failure. Instead, look at the deepest roots of each engine's hierarchy and see what could be unified. That's why I pointed out googletest, TAML, Module, Console, and Asset systems. If the two engines could at least share these, then there is a brighter future for both engines even if they never fully merge.

#15

Quote:For a start, we'd need to port T3D over to 4-spaced code :P.

DO IT! 4 spaces > tab character, forever.

#16

Quote:Without a more explanation of this suggestion and a technical analysis, the cautious side of me will respond with "Severe rectal bleeding" in regard to the level of "pain in the rear." This measurement will change if you go into further details.

Mostly attempting to determine what all could be pulled on out for separate shared tracking in terms of the /core directory and whatnot with an eye for eventually including the higher end things like taml and the like.

#17 @Azaezel - Oh, I see. I would have to take a look at T3D's current codebase. Off the top of my head:

* Console
* ConsoleObject
* SimObject
* Math
* Maybe input
* Maybe SFX

The last too are very iffy. For anything else, a code review of T3D would be needed.

#18 I just realized, in addition to Scaleform-like UIs, we could do interactive puzzle UI elements like in Resident Evil or MoonBase Alpha... and make an Open Source MoonBase Alpha! John Madden! AEIOU! HOLLA HOLLA GET DOLLA!

#19 How do you spot the developer who ingested caffeine most recently?

#20 One other major point of difference between the engines is T2D's entity/component system. This is something T3D has needed for a long time. Jeff Raab is working on one, but AFAIK he's working on it with little knowledge of how T2D's system works. Once his version is open and out there, we should evaluate the differences between them and see which one T3D should move ahead with, and maybe even whether we could make any improvements to T2D's system.

We could also try to manage smaller ports - for example, maybe we could bring sprite-based UI controls into T3D. I imagine that would rely on TAML and so on being merged beforehand.