Torque 3D 3.7 released
by Daniel Buckmaster · 06/24/2015 (2:01 am) · 13 comments
3.7 is out in the wild! We've run a bit into overtime, but the additional work squeaked in at the end was worth it. You can grab a package with the Project Manager and Windows binaries from here, or simply 'git pull' the master branch to get the latest version of the source.
We've seen a lot of work go into 3.7 and it's been a pretty hefty batch of polishing, improvement and fixes, with a dash of new stuff in for flavor. Over 200 issues and pull requests were closed for 3.7, plus another 40 or so which were categorised under the OpenGL milestone. Of all those, around 100 were bug/fixes, 40 were new features, and the rest were general improvements to performance or code quality.
As always, you can see the full list of changes in the release notes, but here are some highlights:
Torque 3D running on top of my XMonad desktop, for extra street cred
As if you haven't heard this enough already, T3D now supports a Linux client with OpenGL rendering. This has been a herculean effort by some dedicated and talented programmers, so everyone give them a huge pat on the back! Particular thanks goes to Luis, who has spearheaded the effort, Lopuska, who has helped hugely, Azaezel, who has been diligent as usual, and BeamNG, who have supported Luis's efforts and the engine in general.
Note that not everything is working quite right in Linux just yet - there's still work to be done, especially around the platform layer, and some graphical bugs that no doubt lurk in different GFX/driver combinations. But this is a huge step towards full cross-platform support for the engine, and we're really happy to have Linux back.
A demonstration of Sahara, from BitGap Games' site
After Konrad Kiss kindly released Sahara under an open-source license, Quinton Delpeche and Andrew Mac set about getting it ready to use with the current version of the engine. Andrew even made some improvements, so you can use multiple different accumulation types in the same level, defined by volumes you can place like triggers. We reckon this is a pretty cool feature, letting you reuse assets while tying your envionments together in a more cohesive style.
Ribbon projectile being tested in Airship Dragoon
Another fun litle graphical upgrade is the new Ribbon class, which can be attached to projectiles as in the image above, or even vehicles. You can make all sorts of crazy things (shudder).
A large navmesh in the new navmesh editor, formerly known as Walkabout
A little while ago, I open-sourced Walkabout, a formerly-commercial addon providing Recast/Detour integration. Thanks to popular demand I've cleaned it up a little and PRed it into the main engine, so everyone can enjoy the power of those fantastic libraries with Torque.
A subtle vignette and a tweaked atmospheric ray effect
One of my personal favourites is that you can now write anonymous functions in TorqueScript. Though they don't capture any surrounding variables, they're still useful for scheduling and callbacks! There's also a new vignette PostFX shader courtesy of J0linar, and an in-depth tweaking interface for the god-ray shader, thanks to Chelaru's first PR. Lukas also managed to get particles rendering to the glow buffer, which means fancy glowing particles!
As always, you can find the full changelog the wiki. But here I want to mention a couple of significant changes that may have implications for your project.
The folks at GarageGames were kind enough to let the SC get access to some of the older Torque 3D demos - so we're really happy that we will soon be presenting them to you, reworked to use the latest engine version and totally open-source! This means that the Pacific, Sector T3D, and Deathball Desert demos will be coming to you soon. Jeff Raab has been taking the responsibility of getting them ported, and we're really excited to be able to give them to you soon.
In the meantime, here are a couple of high resolution screen shots!
Five, sadly, according to our GitHub milestone. I can't remember how long I've been assuring people that it's okay, the release candidate will be out next week! And, after that, that we're just waiting for more tests and to fix the remaining issues before releasing properly. We on the SC will be the first to admit that we've been pretty slow to wrap this one up. What with this and that, and BeamNG's recent release tying up some of our most productive reviewers, we've been a little slow to get through that last 10% of work for the release.
So, with that in mind, let's chat a bit about what you can expect in the next release.
I had to double-check using my calculator, but it tells me that 3.8 would be the next step in line. What's on the docket for 3.8, you ask? We on the Steering Committee have been mulling over stuff we feel is pretty critical to Torque's advancement. A lot of it is the finalization and integration of several projects people in the community have been working on for a while now, while other parts pertaining to cleaning up the weak aspects you guys have to deal with constantly. Specifically:
But now, before I get too far ahead of myself - obviously not all of these are going to make it into 3.8, due to sheer impracticality. One of our biggest issues right now is that we're not doing a great job of providing a stable codebase for you, the users. The mix of lack of testing, huge codebase, large PRs with sweeping changes, and just a general shortage of eyes on the code has meant some projects like DHMC haven't been able to upgrade past 3.5.
This is a big problem for us, and we plan on taking it seriously. Our recent internal discussions have focused on the possibility of making 3.8 a 'boring' release focused on stability and improving our processes. We've also been encouraged to focus on some of the community projects out there, like James's hardware skinning, which will bring large benefits to most users of the engine with minimal pain.
So, as usual, we want to hear from you! Some people in the community we hear from a lot, but others tend not to be involved in the same channels of communication as frequently, and though others do an admirable job passing the message along to us in the SC when they can, we want to hear your opinions! Share your thoughts in the comments or in the forums - what would you like to see us focus on, and what would you like to do with the engine?
In no particular order, here are the people who contributed to this release's GitHub issues - we thank and salute you! If anyone's been left off this list, I apologise in advance - I got lazy and used a script to run through the authors of all closed PRs and issues, but we might have let a few slip through the cracks in odd circumstances!
We've seen a lot of work go into 3.7 and it's been a pretty hefty batch of polishing, improvement and fixes, with a dash of new stuff in for flavor. Over 200 issues and pull requests were closed for 3.7, plus another 40 or so which were categorised under the OpenGL milestone. Of all those, around 100 were bug/fixes, 40 were new features, and the rest were general improvements to performance or code quality.
As always, you can see the full list of changes in the release notes, but here are some highlights:
New stuff
Linux
Torque 3D running on top of my XMonad desktop, for extra street cred
As if you haven't heard this enough already, T3D now supports a Linux client with OpenGL rendering. This has been a herculean effort by some dedicated and talented programmers, so everyone give them a huge pat on the back! Particular thanks goes to Luis, who has spearheaded the effort, Lopuska, who has helped hugely, Azaezel, who has been diligent as usual, and BeamNG, who have supported Luis's efforts and the engine in general.
Note that not everything is working quite right in Linux just yet - there's still work to be done, especially around the platform layer, and some graphical bugs that no doubt lurk in different GFX/driver combinations. But this is a huge step towards full cross-platform support for the engine, and we're really happy to have Linux back.
Accumulation volumes
A demonstration of Sahara, from BitGap Games' site
After Konrad Kiss kindly released Sahara under an open-source license, Quinton Delpeche and Andrew Mac set about getting it ready to use with the current version of the engine. Andrew even made some improvements, so you can use multiple different accumulation types in the same level, defined by volumes you can place like triggers. We reckon this is a pretty cool feature, letting you reuse assets while tying your envionments together in a more cohesive style.
Ribbons
Ribbon projectile being tested in Airship Dragoon
Another fun litle graphical upgrade is the new Ribbon class, which can be attached to projectiles as in the image above, or even vehicles. You can make all sorts of crazy things (shudder).
Navmeshes
A large navmesh in the new navmesh editor, formerly known as Walkabout
A little while ago, I open-sourced Walkabout, a formerly-commercial addon providing Recast/Detour integration. Thanks to popular demand I've cleaned it up a little and PRed it into the main engine, so everyone can enjoy the power of those fantastic libraries with Torque.
Bits and pieces
A subtle vignette and a tweaked atmospheric ray effect
One of my personal favourites is that you can now write anonymous functions in TorqueScript. Though they don't capture any surrounding variables, they're still useful for scheduling and callbacks! There's also a new vignette PostFX shader courtesy of J0linar, and an in-depth tweaking interface for the god-ray shader, thanks to Chelaru's first PR. Lukas also managed to get particles rendering to the glow buffer, which means fancy glowing particles!
Behind-the-scenes
As always, you can find the full changelog the wiki. But here I want to mention a couple of significant changes that may have implications for your project.
- Most uses of ConsoleMethod have been changed to use DefineEngineMethod. The old macros haven't been removed, and are in fact still used for variadic script functions, which aren't currently possible with the DefineEngineMethod macros. But be warned that there are a lot of changes in this area, which may cause conflicts if you've modified any of the console methods.
- James Urquhart's major console function call refactor has finally found its way in, which has meant significant refactoring to the way TorqueScript works under the hood, including new wrappers for console types. This has introduced some subtle bugs we've managed to find, but if you're relying on custom console methods and behaviour, we urge you to double-check those and make sure they still behave exactly as expected.
- A bug in quaternion math has been fixed, but you might find that code relying on the broken behaviour needs to be changed.
- There's a new version of the SimDictionary based on the C++11 STL hashmap. You'll find a commented-out #define in torqueConfig.h which you can enable if you like. According to the contributors, it's worth doing if you have a lot of SimObjects about at the same time.
- The projects.xml file is now part of the main repo, instead of being part of the Project Manager repo. Just so you know, and don't go looking for it there.
- isObject is now stricter about what's actually an object. Where before, the string "3.14" would be cast to the integer 3 and resolved to object ID 3, the whole string is now validated first, and if it's not a valid integer, it won't count as an object ID. (Note that the method still also accepts object names as usual.) This logic expands to other types of object resolution as well - so "3.14".getClassName() now won't work.
Extra goodies
The folks at GarageGames were kind enough to let the SC get access to some of the older Torque 3D demos - so we're really happy that we will soon be presenting them to you, reworked to use the latest engine version and totally open-source! This means that the Pacific, Sector T3D, and Deathball Desert demos will be coming to you soon. Jeff Raab has been taking the responsibility of getting them ported, and we're really excited to be able to give them to you soon.
In the meantime, here are a couple of high resolution screen shots!
How many months overdue?
Five, sadly, according to our GitHub milestone. I can't remember how long I've been assuring people that it's okay, the release candidate will be out next week! And, after that, that we're just waiting for more tests and to fix the remaining issues before releasing properly. We on the SC will be the first to admit that we've been pretty slow to wrap this one up. What with this and that, and BeamNG's recent release tying up some of our most productive reviewers, we've been a little slow to get through that last 10% of work for the release.
So, with that in mind, let's chat a bit about what you can expect in the next release.
What's next
I had to double-check using my calculator, but it tells me that 3.8 would be the next step in line. What's on the docket for 3.8, you ask? We on the Steering Committee have been mulling over stuff we feel is pretty critical to Torque's advancement. A lot of it is the finalization and integration of several projects people in the community have been working on for a while now, while other parts pertaining to cleaning up the weak aspects you guys have to deal with constantly. Specifically:
- Continue merging our massive backlog of pull requests! Seriously, it's hard to keep up with you lot.
- Jeff's entity/component system. This is a massive and important refactoring, which is going to hit like a tsunami.
- Some sort of plugin or package system for C++/scripts/content. This will be supported by
- Improving the Project Manager - specifically, we want to rewrite it as a Torque application, instead of it using Qt, and beef it up with new features.
- Improve Linux and OpenGL work - in particular, stability, and making sure we have feature parity across platforms.
- Refactor ugly/old code. This is a constant war.
- Review the physics layer. Common consensus dictates that the builtin 'Torque physics' shouldn't be privileged, but should be a backend the same way Bullet and PhysX are, meaning no more duplicated RigidShape/PhysicsShape split.
But now, before I get too far ahead of myself - obviously not all of these are going to make it into 3.8, due to sheer impracticality. One of our biggest issues right now is that we're not doing a great job of providing a stable codebase for you, the users. The mix of lack of testing, huge codebase, large PRs with sweeping changes, and just a general shortage of eyes on the code has meant some projects like DHMC haven't been able to upgrade past 3.5.
This is a big problem for us, and we plan on taking it seriously. Our recent internal discussions have focused on the possibility of making 3.8 a 'boring' release focused on stability and improving our processes. We've also been encouraged to focus on some of the community projects out there, like James's hardware skinning, which will bring large benefits to most users of the engine with minimal pain.
So, as usual, we want to hear from you! Some people in the community we hear from a lot, but others tend not to be involved in the same channels of communication as frequently, and though others do an admirable job passing the message along to us in the SC when they can, we want to hear your opinions! Share your thoughts in the comments or in the forums - what would you like to see us focus on, and what would you like to do with the engine?
Acknowledgments
In no particular order, here are the people who contributed to this release's GitHub issues - we thank and salute you! If anyone's been left off this list, I apologise in advance - I got lazy and used a script to run through the authors of all closed PRs and issues, but we might have let a few slip through the cracks in odd circumstances!
- jamesu
- Levitator1
- eightyeight
- Azaezel
- Areloch
- LuisAntonRebollo
- J0linar
- John3
- brakhane
- bpay
- pbrand
- adambeer
- antonyjones67
- MusicMonkey5555
- SteveYorkshire
- JeffProgrammer
- Lopuska
- dragutux
- Packer
- ChelaruCatalin
- rextimmy
- cornytrace
- Winterleaf
- GuyAllard
- lukaspj
- chieling
- skaughtx0r
- qdelpeche
- andr3wmac
- just-bank
- rasteron
- thecelloman
- aurodev
About the author
Studying mechatronic engineering and computer science at the University of Sydney. Game development is probably my most time-consuming hobby!
#2
06/24/2015 (5:02 pm)
So glad to see the engine evolving. Good job guys!
#3
I gave the new release a whirl under Opensuse 13.2. I am using an ATI radeon hd6970. The gui loads and the mission starts but the screen is all one color. I can provide more details but I wanted to give some feedback about Torque 3d. I have been a user for ten years now.
Here are a few thoughts I had about what we could do to improve our usability:
- Simplify the engine. There are roughly 11,000 files that comprise about 152MB. That just seems like way too much. Maybe gut a whole bunch of stuff that is useless and definitely streamline everything. I know for a fact there is a ton of stuff in there that doesn't even work.
- The engine design needs to be cleaner. After 15 years of mods and switching owners the code on sections dealing with graphics drivers, multiple platforms, and displaying graphics is a spaghetti mess;) This is going to require some serious thought and discussion. Along with this, I think there needs to be a high level layout of the current engine design;) This could really help point out where the engine design needs work in order to become easier to use and more useful.
- Place to talk about issues. Right now we have info on the garagegames website, the new wiki torque3d.org, and bug issues on github. This is not very well connected or easily traversed to find problems. A google search is like stabbing yourself in the eye to find a particular problem.
- Not enough feedback/eyes on the project. You used to be able to just start hacking away at the engine and then when you came to hard spot, hit the gg forum and bam somebody already ran into that. This is next to non-existent now. I think this ties in with my last issue which was we need one central spot in one format that is easily searched and/or traversed.
- Driver issues for "linux." Linux is not just one operating system called ubuntu:) Linux comes in so many flavors:) However, there is a standard mesa driver that will be setup on most linux distros by default. We should really support this first and foremost even if it only can support the most basic lighting etc.
- Get your priorities straight. New cool features that only work on a windows box with a certain driver or video card are useless to 90 percent of users of the game engine. We really need to have a basic level of operation. Our goal should be to actually have the engine and editors work across all platforms even at the most basic level of displaying objects, animating objects, audio, particles, etc.. Yeah that may not be cool or fun but it is more realistic if you want to provide and engine that is actually useful. Many game developers are looking to push there games across many platforms including phones and tablets. Unity crushed torque a long time ago because they were able to support so many platforms.
- I really think torque should focus on the basics. You have to be able to walk or even stand before you can run:) Even though it may work okay on windows it doesn't mean anything when most young kids today don't even know how to get on a desktop or even ever owned one. You can download your favorite distro of linux for free.
I know I opened up serious cans of worms there but I think it is good for getting the discussion started.
06/24/2015 (9:53 pm)
First off, thanks for everyone that is working hard on this! I gave the new release a whirl under Opensuse 13.2. I am using an ATI radeon hd6970. The gui loads and the mission starts but the screen is all one color. I can provide more details but I wanted to give some feedback about Torque 3d. I have been a user for ten years now.
Here are a few thoughts I had about what we could do to improve our usability:
- Simplify the engine. There are roughly 11,000 files that comprise about 152MB. That just seems like way too much. Maybe gut a whole bunch of stuff that is useless and definitely streamline everything. I know for a fact there is a ton of stuff in there that doesn't even work.
- The engine design needs to be cleaner. After 15 years of mods and switching owners the code on sections dealing with graphics drivers, multiple platforms, and displaying graphics is a spaghetti mess;) This is going to require some serious thought and discussion. Along with this, I think there needs to be a high level layout of the current engine design;) This could really help point out where the engine design needs work in order to become easier to use and more useful.
- Place to talk about issues. Right now we have info on the garagegames website, the new wiki torque3d.org, and bug issues on github. This is not very well connected or easily traversed to find problems. A google search is like stabbing yourself in the eye to find a particular problem.
- Not enough feedback/eyes on the project. You used to be able to just start hacking away at the engine and then when you came to hard spot, hit the gg forum and bam somebody already ran into that. This is next to non-existent now. I think this ties in with my last issue which was we need one central spot in one format that is easily searched and/or traversed.
- Driver issues for "linux." Linux is not just one operating system called ubuntu:) Linux comes in so many flavors:) However, there is a standard mesa driver that will be setup on most linux distros by default. We should really support this first and foremost even if it only can support the most basic lighting etc.
- Get your priorities straight. New cool features that only work on a windows box with a certain driver or video card are useless to 90 percent of users of the game engine. We really need to have a basic level of operation. Our goal should be to actually have the engine and editors work across all platforms even at the most basic level of displaying objects, animating objects, audio, particles, etc.. Yeah that may not be cool or fun but it is more realistic if you want to provide and engine that is actually useful. Many game developers are looking to push there games across many platforms including phones and tablets. Unity crushed torque a long time ago because they were able to support so many platforms.
- I really think torque should focus on the basics. You have to be able to walk or even stand before you can run:) Even though it may work okay on windows it doesn't mean anything when most young kids today don't even know how to get on a desktop or even ever owned one. You can download your favorite distro of linux for free.
I know I opened up serious cans of worms there but I think it is good for getting the discussion started.
#4
That's an awesome amount of work!
Came back a couple of months ago and I am really impressed with the progress that's been made since going MIT. We'll get linux on track.
Torque is looking good these days. I'm glad to be back.
Thanks for the mention!
Aurodev
https://github.com/aurodev
http://forums.torque3d.org/memberlist.php?mode=viewprofile&u=174
06/25/2015 (7:59 pm)
Wow! /gratzThat's an awesome amount of work!
Came back a couple of months ago and I am really impressed with the progress that's been made since going MIT. We'll get linux on track.
Torque is looking good these days. I'm glad to be back.
Thanks for the mention!
Aurodev
https://github.com/aurodev
http://forums.torque3d.org/memberlist.php?mode=viewprofile&u=174
#6
First, I want to say I agree with all of your points, before I go into more detail. I think you've basically summarised my personal view on what we need to do as a community.
06/26/2015 (12:32 am)
Kyle: thanks for the feedback! It's great to hear people's opinions on this sort of stuff. Ultimately our job is to make T3D the best it can be, and support its current users, so the more datapoints we have on the issues you have, the better.First, I want to say I agree with all of your points, before I go into more detail. I think you've basically summarised my personal view on what we need to do as a community.
Quote:Simplify the engine.Definitely needs to happen. In our defence, the codebase is far smaller than something like UE or Unity, but it's likely also less well-documented and structured. Removing old code needs to happen, I agree, as well as more comprehensive documentation of the parts that *do* work and *are* there.
Quote:The engine design needs to be cleaner.No disagreements, but as you said, it's going to be a heck of an effort. This is something we are very concerned with in the SC and I'm sure discussion has been had in the community, too.
Quote:Place to talk about issues.This has been a problem for as long as I can remember, and is also a problem these days with less people using the engine (so you're less likely to run into someone doing the same thing or having the same problem). The places we have now are the GitHub issues page (for submitting bug reports and discussing proposed merges), forum.torque3d.org (for general ideas, help, etc.), wiki.torque3d.org (for documentation, tutorials), and here, which is pretty much now a legacy database with lots of information that's not very accessible. How we might consolidate these sources is an open question.
Quote:Not enough feedback/eyes on the project.True but not something we can directly solve.
Quote:Linux is not just one operating system called ubuntu:)Good point. I guess we should clarify in our communication that 'Linux support' means 'a Linux support', not 'all Linux support' :p. I've tried to be inclusive - for example, on the wiki page about compiling, there are instructions for as many varieties of Linux as I am aware of instructions for. I agree we should keep this as well as possible, but it's just an issue of effort. We don't all run multiple flavours of Linux to test/develop on so it's just an issue of having more (and more diverse) eyes/keyboards on the project.
Quote:I really think torque should focus on the basics.I agree. Though, actually, I completely disagree that we should focus on Linux for accessibility reasons. Young kids today haven't got a clue how to download Linux, nor a favourite distribution :p.
#7
Thanks for the link:)
@ Daniel
Just wanted to share my two cents:) Sorry if it came off as too harsh. If you have a stable minimal version for linux, then you have something that can work on android, mac, and even iphone/ipad. These are the platforms that everyone is using.
Thanks everyone again for all your hard work and dedication!
06/26/2015 (5:08 pm)
@Timmy Thanks for the link:)
@ Daniel
Quote:So, as usual, we want to hear from you! Some people in the community we hear from a lot, but others tend not to be involved in the same channels of communication as frequently, and though others do an admirable job passing the message along to us in the SC when they can, we want to hear your opinions! Share your thoughts in the comments or in the forums - what would you like to see us focus on, and what would you like to do with the engine?
Just wanted to share my two cents:) Sorry if it came off as too harsh. If you have a stable minimal version for linux, then you have something that can work on android, mac, and even iphone/ipad. These are the platforms that everyone is using.
Thanks everyone again for all your hard work and dedication!
#8
Thank you all who have spent their precious time into this!
Hopefully this will trigger a load of newcomers ;-)
06/28/2015 (7:35 pm)
Awesome stuff!!! Thank you all who have spent their precious time into this!
Hopefully this will trigger a load of newcomers ;-)
#9
07/01/2015 (3:54 am)
Awesome. It takes a lot of dedication to keep the boat afloat, huzzah for all you brave warriors. 
#10
Thank you very much!
07/02/2015 (12:54 am)
Its great you guys are still improving this massive beast.Thank you very much!
#11
Only problem is. It's just ten years too late. I get the overwhelming sense that if Linux had been developed for properly way back when TGEA was the first attempt at the new rendering system, then the whole code base would be in much better shape.
But hey, I've been in and out of the scene since T3D was first released as a Windows only engine, and was pretty darn disappointed at the design decisions. I pretty much ended up dropping my whole game idea and start-up after that happened.
Might have another look-see at the engine.
07/11/2015 (7:55 pm)
Man I am dead glad that others put in the effort for the Linux version. Excellent work guys.Only problem is. It's just ten years too late. I get the overwhelming sense that if Linux had been developed for properly way back when TGEA was the first attempt at the new rendering system, then the whole code base would be in much better shape.
But hey, I've been in and out of the scene since T3D was first released as a Windows only engine, and was pretty darn disappointed at the design decisions. I pretty much ended up dropping my whole game idea and start-up after that happened.
Might have another look-see at the engine.
#12
Thanks for sticking to this tech and making it more and more awesome version after version!
... shamelessly downloading the version right now .... Muhahahahaha!
07/14/2015 (11:13 am)
This version got to be the most hot of them all. Frankly speaking I am stunned with all the awesome stuff this version is loaded with :O)Thanks for sticking to this tech and making it more and more awesome version after version!
... shamelessly downloading the version right now .... Muhahahahaha!
#13
08/25/2015 (12:46 pm)
Well done to everyone participating in the development and Thank You to those that are merging their own open-soured projects, you guys are giving some needed horsepower to Torque! 
Torque Owner Kyle Hagin
B + Games