Torque 3D Status - 7/29/2010
by Matt Fairfax · in Torque 3D Professional · 07/29/2010 (8:02 am) · 111 replies
Hey all,
Let's start with the good news =)
We are fairly confident that we are going to be able to deliver Torque 3D 1.1 Beta 2 in the next couple of days. We have a few more blocker bugs to work through before we will release it and the Docs team is jamming long, long hours to make sure that it ships with a complete script reference. As always, there is risk that something unexpected will pop up and push us into the beginning of next week but we are working hard to make sure that doesn't happen.
After I post this, I am going to be posting a preliminary changelog so that you can start to wrap your head around just how much work has gone into this release.
Here is some statistics that I hope you guys will find very exciting:
New Features - 21
Improvements - 191
Fixes - 254
Now the bad news...
Torque X 3.1.5 took a bit longer than we had hoped and we didn't really begin to ramp QA back up onto Torque 3D fulltime until last week. We have been working hard to capture a complete picture of exactly what is left to fix in Torque 3D 1.1 and the list is longer than we'd like despite the huge strides we have made (as you will see in the changelog). There is still a lot more bugs we want to fix before we call this 1.1 Final...so many so that we think we will need a Beta 3.
Now, before anyone gets too concerned...there will be a much shorter gap between Beta 2 and Beta 3 than there was between Beta 1 and Beta 2. By shorter, I mean closer to a month rather than 5 months. I will also be updating you guys weekly on what we have fixed and what new bugs we have discovered until we ship Beta 3.
Tomorrow, I plan to post a Known Issues list showing the bugs that we will be focusing on for Beta 3. This list isn't completely fleshed out yet because there are still bugs making their way through the QA pipeline that haven't been properly verified, documented, and prioritized for the developers. There are a lot of bug reports on the Torque 3D forums which is an amazing testament to how awesome this community is and we are still playing some catch up. As I mentioned above, this Known Issues list will get updated every week and I will also be adding some lower priority bugs as I get them organized.
There is a slight chance that I won't be able to get the Known Issues list posted tomorrow because I have some family business that could interfere but I will try my best!
Disclaimer: All timeframes in this thread are estimates and should not be considered final
Let's start with the good news =)
We are fairly confident that we are going to be able to deliver Torque 3D 1.1 Beta 2 in the next couple of days. We have a few more blocker bugs to work through before we will release it and the Docs team is jamming long, long hours to make sure that it ships with a complete script reference. As always, there is risk that something unexpected will pop up and push us into the beginning of next week but we are working hard to make sure that doesn't happen.
After I post this, I am going to be posting a preliminary changelog so that you can start to wrap your head around just how much work has gone into this release.
Here is some statistics that I hope you guys will find very exciting:
New Features - 21
Improvements - 191
Fixes - 254
Now the bad news...
Torque X 3.1.5 took a bit longer than we had hoped and we didn't really begin to ramp QA back up onto Torque 3D fulltime until last week. We have been working hard to capture a complete picture of exactly what is left to fix in Torque 3D 1.1 and the list is longer than we'd like despite the huge strides we have made (as you will see in the changelog). There is still a lot more bugs we want to fix before we call this 1.1 Final...so many so that we think we will need a Beta 3.
Now, before anyone gets too concerned...there will be a much shorter gap between Beta 2 and Beta 3 than there was between Beta 1 and Beta 2. By shorter, I mean closer to a month rather than 5 months. I will also be updating you guys weekly on what we have fixed and what new bugs we have discovered until we ship Beta 3.
Tomorrow, I plan to post a Known Issues list showing the bugs that we will be focusing on for Beta 3. This list isn't completely fleshed out yet because there are still bugs making their way through the QA pipeline that haven't been properly verified, documented, and prioritized for the developers. There are a lot of bug reports on the Torque 3D forums which is an amazing testament to how awesome this community is and we are still playing some catch up. As I mentioned above, this Known Issues list will get updated every week and I will also be adding some lower priority bugs as I get them organized.
There is a slight chance that I won't be able to get the Known Issues list posted tomorrow because I have some family business that could interfere but I will try my best!
Disclaimer: All timeframes in this thread are estimates and should not be considered final
About the author
I am a Game Designer at PopCap who has worked on PvZ Adventures, PvZ2, Peggle Blast, and Bejeweled Skies. I am an ex-GarageGames employee who helped ship TGE, TGEA, Torque 3D, and Constructor.
#22
07/29/2010 (6:13 pm)
Wow! Looks like many of the irritating issues we have been addressing have been solved for us, Awesome! Damn, Guess this is gonna push the start of our testing back another week... Lol 
#23
So, everyone please begin using this standard for all future bug reports. Adhering to this standard will be an essential part of our goal to decrease the time between forum bug logged, bug verified, and bug passed to the programmers.
07/29/2010 (9:20 pm)
Everyone's probably noticed by the thread being at the top of all the T3D bug forums, but I've gone ahead and added our new bug reporting standard to T3D. Typically I've waited for the next release of each engine before doing this, but the sooner we start using it here the better.So, everyone please begin using this standard for all future bug reports. Adhering to this standard will be an essential part of our goal to decrease the time between forum bug logged, bug verified, and bug passed to the programmers.
#25
2 other things, very simple, but takes discipline:
Make as little or NO changes to the engine source (or core .cs) files, and if you do, mark *every one* clearly with an easily searchable moniker, ie:
// JAMESONCHANGE -->
...CODE
// <-- JAMESONCHANGE
All other new classes, or additional class code I add in to the project source folder, never to the engine folders. My main intent is to keep the source as clean as possible.
My game files (.cs/art/etc) are significantly different than the stock at this point so I dont bother 'merging' those, although with each release I do skim thru to make sure there werent any changes to the way the game starts up.
I also avoid adding in resources... I use them to study, but have been bitten a couple times by small bugs introduced by copy/pasted code I had not really dug into and did not understand, or missed part of in a merge. All non-stock T3D in my app I've typed in myself and I follow the rules above.
A merge then simply becomes: move project folders, search for the moniker and migrate the needed code. compile. fix compilation bugs (most are usally moved files, some functions usually change in an obvious way). Run the app.
Since I've done this, I don't think I've cracked open a merge app to migrate. My last merge took less than 10 mins on a clean T3D install, with about 90,000 LOC I've added to the engine.
Granted, for the code side, it's pretty much just me, along with 7+ years with the TGE/T3D code kicking my ass to get me into this habit, although I've only been developing a personal game with it for 10 months. Forcing the discipline for this on a team is a whole different matter. I've tried several times as a project manager, and never succeeded; everyone has their own coding style.
07/30/2010 (12:40 am)
@Caylo: ditto what Tom said, that app is awesome. Winmerge aint too shabby either, gotta give props. Even with a good merge tool, if the update has a lot of little fixes, it can take a LONG time of concentrated effort to ensure it all gets migrated.2 other things, very simple, but takes discipline:
Make as little or NO changes to the engine source (or core .cs) files, and if you do, mark *every one* clearly with an easily searchable moniker, ie:
// JAMESONCHANGE -->
...CODE
// <-- JAMESONCHANGE
All other new classes, or additional class code I add in to the project source folder, never to the engine folders. My main intent is to keep the source as clean as possible.
My game files (.cs/art/etc) are significantly different than the stock at this point so I dont bother 'merging' those, although with each release I do skim thru to make sure there werent any changes to the way the game starts up.
I also avoid adding in resources... I use them to study, but have been bitten a couple times by small bugs introduced by copy/pasted code I had not really dug into and did not understand, or missed part of in a merge. All non-stock T3D in my app I've typed in myself and I follow the rules above.
A merge then simply becomes: move project folders, search for the moniker and migrate the needed code. compile. fix compilation bugs (most are usally moved files, some functions usually change in an obvious way). Run the app.
Since I've done this, I don't think I've cracked open a merge app to migrate. My last merge took less than 10 mins on a clean T3D install, with about 90,000 LOC I've added to the engine.
Granted, for the code side, it's pretty much just me, along with 7+ years with the TGE/T3D code kicking my ass to get me into this habit, although I've only been developing a personal game with it for 10 months. Forcing the discipline for this on a team is a whole different matter. I've tried several times as a project manager, and never succeeded; everyone has their own coding style.
#26
Also - it takes balls to push in another beta and be so honest about QA. You're doing things like a rock star, finally!
Way to go, Torque!
07/30/2010 (12:42 am)
Great work guys! That is an awesome list. It looks like you've worked an entire year or two within those 5 months. :)Also - it takes balls to push in another beta and be so honest about QA. You're doing things like a rock star, finally!
Way to go, Torque!
#27
For me is was Beyond Compare as well. Any sort of auto merging ended in a disaster always. I also liked to have two copies of the original when I started so that I could do three-way views of: code A, merged code, and code B. That made things go quicker after a week of merging.
Practice makes perfect, but it's a great way to hone your debugging skills and learn an engine.
07/30/2010 (12:59 am)
@Caylo - I was a full time merge monkey many jobs ago. I once merged a year old R&D branch with a full on productive branch. It was a mess.For me is was Beyond Compare as well. Any sort of auto merging ended in a disaster always. I also liked to have two copies of the original when I started so that I could do three-way views of: code A, merged code, and code B. That made things go quicker after a week of merging.
Practice makes perfect, but it's a great way to hone your debugging skills and learn an engine.
#28
------------
On the subject of porting/upgrading, I've written myself a porting guide (just scrawled over a couple of pieces of A4 - saved hours using it to port to 1.1.b1 and a lot less stressful), and I've also started commenting bugfixes, 'cos sometimes I looked at something and wondered what the hell I had altered it for ... //yorks - bugfix for xyz ... //yorks changed - stock was xyz ... //yorks - new
07/30/2010 (1:18 am)
@KonradQuote:Waking up covered in their own sick in a trashed motel room? Well, I guess that'll explain the wait ... ;o)
You're doing things like a rock star, finally!
------------
On the subject of porting/upgrading, I've written myself a porting guide (just scrawled over a couple of pieces of A4 - saved hours using it to port to 1.1.b1 and a lot less stressful), and I've also started commenting bugfixes, 'cos sometimes I looked at something and wondered what the hell I had altered it for ... //yorks - bugfix for xyz ... //yorks changed - stock was xyz ... //yorks - new
#29
It would be great if you can share the "porting guide" you have written for yourself... ;)
07/30/2010 (2:52 am)
Steve !It would be great if you can share the "porting guide" you have written for yourself... ;)
#30
07/30/2010 (3:26 am)
Very impressive list of fixes and improvements. The console updates look really great. Go Rene!! 
#31
I have done little in regards to T3D C'code modifications yet for this very reason. Only experimental proofs, holding out in anticipation of some day (soon now, I can feel it in my TorquePowered bones!) bringing all my scraps together.
But now with the growing desire to be able to port my project over to TX3D and perhaps iTorque if it is feasible, I am trying to think of the most effective way to program my project with portability in mind.
07/30/2010 (3:54 am)
Thanks for the great ideas on the subject of making porting and merging code tasks less painful and time consuming. Some real good ideas I had not thought of.I have done little in regards to T3D C'code modifications yet for this very reason. Only experimental proofs, holding out in anticipation of some day (soon now, I can feel it in my TorquePowered bones!) bringing all my scraps together.
But now with the growing desire to be able to port my project over to TX3D and perhaps iTorque if it is feasible, I am trying to think of the most effective way to program my project with portability in mind.
#32
07/30/2010 (7:14 am)
Sorry all...I wasn't able to finish up the Known Issues list today. I'll make sure I get it up tomorrow (probably late).
#33
It's just a very long list of script files which I have altered or created, with an extra list of code alterations (most of my big code changes are whole cusotm files which should just drop in) and bugfixes to check for. As I comment my changes (especially in code where I'm not as comfortable as script) with an easy searchable tag like Jameson mentions above (//yorks), it makes finding and comparing them all the more easy.
I'd recommend that anyone who, like me, was previously struck by the fear of god when it came to porting, take the time to list and record their process for moving their custom project over into a clean New Project. Makes things infinitely less stressful in the future.
07/30/2010 (11:30 am)
@SaileshIt's just a very long list of script files which I have altered or created, with an extra list of code alterations (most of my big code changes are whole cusotm files which should just drop in) and bugfixes to check for. As I comment my changes (especially in code where I'm not as comfortable as script) with an easy searchable tag like Jameson mentions above (//yorks), it makes finding and comparing them all the more easy.
I'd recommend that anyone who, like me, was previously struck by the fear of god when it came to porting, take the time to list and record their process for moving their custom project over into a clean New Project. Makes things infinitely less stressful in the future.
Quote:Tremors:
See, we plan ahead, that way we don't do anything right now.
#34
07/30/2010 (12:07 pm)
Can't wait to get my hands on the new release chaps. Big congratulations on the massive update. 8() 
#35
1) identify every source code changes you make using BEGIN and END comment blocks.
2) When replacing existing code, keep the original code commented (ideally inside a comment block). It makes it much easier to merge.
3) Use SVN/GIT/Mercurial!
Here's my SVN strategy for merging new T3D versions:
1) Put the unmodified T3D version your project is currently using in a versioned folder.
2) Make two copies of it. Call one "theirs" and the other "mine".
3) Use WinMerge to copy your project's folder into "mine" (only source code! Merge scripts separately - it's much easier).
4) Use WinMerge to copy the new beta into "theirs" (again, only source).
(I use WinMerge because it will deal with deleted/moved files).
5) Commit "theirs". This will create a revision that contains exactly all changes from the previous T3D to beta2.
6) Merge "theirs" into "mine" (or just update "mine", if both folders point to the same location). "Mine" contains all the changes from previous T3D to your project, so it makes it much easier to see which files you modified, and where.
7) Solve the conflicts and check if all merges went smoothly (you only need to check the files you really modified).
07/30/2010 (12:38 pm)
Merging is much easier if you proceed like Jameson said:1) identify every source code changes you make using BEGIN and END comment blocks.
2) When replacing existing code, keep the original code commented (ideally inside a comment block). It makes it much easier to merge.
3) Use SVN/GIT/Mercurial!
Here's my SVN strategy for merging new T3D versions:
1) Put the unmodified T3D version your project is currently using in a versioned folder.
2) Make two copies of it. Call one "theirs" and the other "mine".
3) Use WinMerge to copy your project's folder into "mine" (only source code! Merge scripts separately - it's much easier).
4) Use WinMerge to copy the new beta into "theirs" (again, only source).
(I use WinMerge because it will deal with deleted/moved files).
5) Commit "theirs". This will create a revision that contains exactly all changes from the previous T3D to beta2.
6) Merge "theirs" into "mine" (or just update "mine", if both folders point to the same location). "Mine" contains all the changes from previous T3D to your project, so it makes it much easier to see which files you modified, and where.
7) Solve the conflicts and check if all merges went smoothly (you only need to check the files you really modified).
#36
07/30/2010 (1:08 pm)
Any serious development already involves using some sort of version control system, so merging is not an issue. If somebody edit sources without VCS should start to do it ASAP =) My personal choice is Git + GitExtensions as a fabulous windows frontend
#37
07/30/2010 (2:05 pm)
Outstanding work everyone! Great write up Matt.
#38
I keep 4 separate repos for my project:
Engine (mod'd bug fixes, only the changed files)
Source (My files)
Game (exe, final game exported packaged from here)
Art (actually this is a few different repos depending on who is working on what, to keep checkout/update time down)
You COULD keep them all in one repo, but I find that checkouts quickly grow out of control esp if you involve the working art files.
When migrating to a clean T3D install, I just check out on top of the current engine directory, and SVN warns me of the conflicts. I can just worry about those files at that point.
I use php scripts, adapted from the T3D packaging system, to build the release based on the HEAD or specific rev no. Works very well.
Having an auto build/smoke test has also saved me many times and is vital on keeping a clean codebase. Not as much as a single developer (although it is helpful), but when managing development projects with many different programmers, I set up a build system that audibly warned if a build failed using a voice app, saying something like "Somebody poisoned the waterhole!" that could be heard around the office. The shame of breaking a build quickly got people to test thoroughly before committing.
07/30/2010 (2:12 pm)
Great post Manoel, didnt get into the SVN/Git thing, although ABSOLUTELY necessary for any real dev. For my SVN engine repo, I only have the engine source files that I've modified committed. I keep 4 separate repos for my project:
Engine (mod'd bug fixes, only the changed files)
Source (My files)
Game (exe, final game exported packaged from here)
Art (actually this is a few different repos depending on who is working on what, to keep checkout/update time down)
You COULD keep them all in one repo, but I find that checkouts quickly grow out of control esp if you involve the working art files.
When migrating to a clean T3D install, I just check out on top of the current engine directory, and SVN warns me of the conflicts. I can just worry about those files at that point.
I use php scripts, adapted from the T3D packaging system, to build the release based on the HEAD or specific rev no. Works very well.
Having an auto build/smoke test has also saved me many times and is vital on keeping a clean codebase. Not as much as a single developer (although it is helpful), but when managing development projects with many different programmers, I set up a build system that audibly warned if a build failed using a voice app, saying something like "Somebody poisoned the waterhole!" that could be heard around the office. The shame of breaking a build quickly got people to test thoroughly before committing.
#39
07/30/2010 (4:19 pm)
Quote:Now that is awesome, I like it!
I set up a build system that audibly warned if a build failed using a voice app, saying something like "Somebody poisoned the waterhole!" that could be heard around the office. The shame of breaking a build quickly got people to test thoroughly before committing.
#40
We are still several hours away from having the documentation complete (I don't think Mich or Dave have slept more than a couple of hours this week).
If we were to release it today or even tomorrow, we wouldn't be able to give it a final proper QA pass. We'd rather not do that even though it means missing the "July" target we set a few weeks ago.
Instead, we are going to have Fullsail test it over the weekend and plan to release it to you guys late Monday.
07/31/2010 (1:55 am)
Hey all,We are still several hours away from having the documentation complete (I don't think Mich or Dave have slept more than a couple of hours this week).
If we were to release it today or even tomorrow, we wouldn't be able to give it a final proper QA pass. We'd rather not do that even though it means missing the "July" target we set a few weeks ago.
Instead, we are going to have Fullsail test it over the weekend and plan to release it to you guys late Monday.
Torque 3D Owner Caylo Gypsyblood
I seen enough bug fixes and improvements to keep me occupied long into the future of many nights. The improvements are above and beyond anticipation, and what could be defined as necessary to fulfill T3D expectations. Kudos for going that extra distance!
This is the type of open blunt honesty about 'the way things went down' I am excited to see. Product brand loyalty and pride returning... Ah there is that happy feeling I have been missing...
@Jameson, It would be interesting to learn some of your tricks for managing code merging. Perhaps when you have time a blog or simple forum post about such details? Code merging is my C'code achilles heel, with my C'code skills what are already abysmal, I flinch every time I launch WinDiff.