Torque 3D's Achilles heel
by Konrad Kiss · in Torque 3D Professional · 03/24/2012 (11:49 am) · 184 replies
Recently I spent some time getting myself familiar with static code analysis and the more common (indie-budget freindly) tools associated with the topic. #AltDevBlogADay had several posts about such tools by Bruce Dawson and John Carmack. Definitely worth checking out.
So, it all began with me having a very elusive bug somewhere in code. I hoped that it would be something that a static code analyzer would pick up, so I lined up what I could from trial versions to open source software, and got some amazing results once these went through the Torque 3D 1.2 codebase.
I can't remember if the actual problem was found by these tools (it's been a while ago). What I recall is being taken aback by the results I got. But first a disclaimer...
I've been reviewing engine code changes for the past 4 years. Torque 3D is huge. One of the most surprising thing was the fact, that these code analyzers could - proportionally - find only a few problems. However, the few they did find were hard to see even when I knew the problem.
The kinds of problems I found primarily were:
- Uninitialized member variables used without initialization
- Checking the existence of an object and using the same object outside the check condition
- Passing local pointers that could become invalid
- Using && instead of &, || instead of | operations
- Bad casts for variables while the code assumes another cast (ie. U32 vs S32 - especially when negative values are used for error codes - common in some of the linked libraries)
- Typos, cut & paste programming mistakes that compile (probably the most vicious of all)
I'm only going to list my favorites, hoping that this will be a big enough motivation for everyone to start using some basic form of static code analysis. The complete list would be way too long for me to list here.
gfxD3D9TextureManager: using boolean evaluation against variables that were clearly intended to be bitmasks.
guiIconButtonCtrl.cpp: find the highlighted button, I dare you
processedFFMaterial.cpp: making sure (though what about blendDest?)
mRectF.cpp: intersectTriangle
winDInputDevice.cpp: POV_down, not much of a choice (twice in the same file)
rectClipper: clipRect (near eof)
win32Window.cpp: equilibrium
Finally - I need to clarify - by all means, no disrespect was intended towards Torque 3D developers. I would have done far worse most likely. I can't count the many new things in the source that made me go "Wow" and "Yey" - it is clearly visible that the team is bigger and better at what they do. Torque 1.2 is the best Torque yet - and I mean it by comparing the overall usability, the number of bugs, etc..
With this post I'm only trying to point at a solution that can save everyone from such blatant bugs in the future, while - perhaps - also nudging GG a little to take a step in the right direction. ;)
Cheers
--Konrad
So, it all began with me having a very elusive bug somewhere in code. I hoped that it would be something that a static code analyzer would pick up, so I lined up what I could from trial versions to open source software, and got some amazing results once these went through the Torque 3D 1.2 codebase.
I can't remember if the actual problem was found by these tools (it's been a while ago). What I recall is being taken aback by the results I got. But first a disclaimer...
I've been reviewing engine code changes for the past 4 years. Torque 3D is huge. One of the most surprising thing was the fact, that these code analyzers could - proportionally - find only a few problems. However, the few they did find were hard to see even when I knew the problem.
The kinds of problems I found primarily were:
- Uninitialized member variables used without initialization
- Checking the existence of an object and using the same object outside the check condition
- Passing local pointers that could become invalid
- Using && instead of &, || instead of | operations
- Bad casts for variables while the code assumes another cast (ie. U32 vs S32 - especially when negative values are used for error codes - common in some of the linked libraries)
- Typos, cut & paste programming mistakes that compile (probably the most vicious of all)
I'm only going to list my favorites, hoping that this will be a big enough motivation for everyone to start using some basic form of static code analysis. The complete list would be way too long for me to list here.
gfxD3D9TextureManager: using boolean evaluation against variables that were clearly intended to be bitmasks.
if (retTex->mProfile->isRenderTarget() && mslevel != 0 && (mDeviceCaps.Caps2 && D3DDEVCAPS2_CAN_STRETCHRECT_FROM_TEXTURES))
guiIconButtonCtrl.cpp: find the highlighted button, I dare you
ColorI fontColor = mActive ? (highlight ? mProfile->mFontColor : mProfile->mFontColor) : mProfile->mFontColorNA;
processedFFMaterial.cpp: making sure (though what about blendDest?)
result.blendSrc = GFXBlendOne;
result.blendSrc = GFXBlendOne;mRectF.cpp: intersectTriangle
if(contains(a) || contains(b) || contains(b))
winDInputDevice.cpp: POV_down, not much of a choice (twice in the same file)
newEvent.objInst = ( objInst == 0 ) ? SI_DPOV : SI_DPOV
rectClipper: clipRect (near eof)
out_rRect.extent.x = bottomR.x - out_rRect.point.x + 1; out_rRect.extent.x = bottomR.y - out_rRect.point.y + 1;
win32Window.cpp: equilibrium
newLeft = mClamp(newLeft, 0, newLeft);
newTop = mClamp(newLeft, 0, newTop);Finally - I need to clarify - by all means, no disrespect was intended towards Torque 3D developers. I would have done far worse most likely. I can't count the many new things in the source that made me go "Wow" and "Yey" - it is clearly visible that the team is bigger and better at what they do. Torque 1.2 is the best Torque yet - and I mean it by comparing the overall usability, the number of bugs, etc..
With this post I'm only trying to point at a solution that can save everyone from such blatant bugs in the future, while - perhaps - also nudging GG a little to take a step in the right direction. ;)
Cheers
--Konrad
About the author
http://about.me/konrad.kiss
#2
03/24/2012 (12:36 pm)
Someone else had a running list of bad logic, type conversions, inconsistencies, and other such errors a while back. Said post had indeed prompted me to use code analysis tools to spot things like these - it's great to see more and more people becoming aware of them. While conceivably not of the highest priority as compared against crippling issues, at some point GarageGames does need to focus on cleaning up these obscure and annoying problems.
#3
03/24/2012 (12:43 pm)
Well, it is a work in progress. I tried to list the most dangerous ones of the issues I saw, but I'm sure everyone would greatly value it if you shared details about problems you find. Bugs can only be corrected if the team knows about them. That too was the intention of this post. I think the developers of Torque 3D are doing a great job. Especially ever since GG actually started making games with the engine - instead of demos.
#4
I am hearing from others that converting to C# rather than Torque Script is speeding up the engine by quite a bit. If this is significant then it might make sense to continue down the path of making it so Torque can switch out scripting engines.
03/24/2012 (12:55 pm)
Can code analysis be used to determine how much the string evaluation of scripting calls eats up the cycles? I understand a lot of string lookup in being used when the script is executing. Or does that fall under the realm of benchmarking?I am hearing from others that converting to C# rather than Torque Script is speeding up the engine by quite a bit. If this is significant then it might make sense to continue down the path of making it so Torque can switch out scripting engines.
#5
03/24/2012 (12:56 pm)
Oh, I agree. Unfortunately I don't recall any employee interaction on that thread I had in mind... was trying to find it in order to link to it, but evidently I didn't bookmark it and it's lost somewhere in time and horrible forum search.
#6
Edit: Check the use of ProfilerEnable(true/false) and ProfileDump() or profilerDumpToFile( "filename" ) console's functions.
03/24/2012 (1:23 pm)
@Frank, profiling tools are required. The torque is able to give some estimates on how many times a function are called and the time used from the beginning to the end of a piece of code, but are not sufficient information. For a complete analysis must have the right tools. Only in this way you can actually measure the time and processor utilization in great detail and with all the necessary information.Edit: Check the use of ProfilerEnable(true/false) and ProfileDump() or profilerDumpToFile( "filename" ) console's functions.
#7
Pardon me.
03/24/2012 (4:08 pm)
Oops... I feel like I just walked into the ladies' restroom. I'll get back to the artists section now.Pardon me.
#8
03/24/2012 (4:18 pm)
lol
#9
Ill admit, changing from TorqueScript to C# has made things run faster and smoother. However, since we dont have profiling tools to say HOW MUCH maybe i should add "to the naked eye".
On the same note, once we can get a few more programmers to help, there are more plans to try to boost performance. We are wanting to dig into the C++ itself and do a bit of a rewrite. We feel its time to bring T3D up to speed.
03/24/2012 (4:46 pm)
@Alfio -what profiling tools would you suggest? Im part of the group thats Frank's talking about. Ill admit, changing from TorqueScript to C# has made things run faster and smoother. However, since we dont have profiling tools to say HOW MUCH maybe i should add "to the naked eye".
On the same note, once we can get a few more programmers to help, there are more plans to try to boost performance. We are wanting to dig into the C++ itself and do a bit of a rewrite. We feel its time to bring T3D up to speed.
#10
Once you have removed the biggest mistakes, you go to profiling. I normally use the visual studio analyzer, or the tools made available by the various platform SDK (see the Sony SDK or Microsoft XDK)
But in the case of Torque3D with the vs analyzer, it always crash advising me to simplify the code. I think it's a problem with the new VS2010.
Should i try with some other software for profiling (better if free).
03/24/2012 (7:40 pm)
First of all would fix part of the code, perhaps with the aid of a static analyzer.Once you have removed the biggest mistakes, you go to profiling. I normally use the visual studio analyzer, or the tools made available by the various platform SDK (see the Sony SDK or Microsoft XDK)
But in the case of Torque3D with the vs analyzer, it always crash advising me to simplify the code. I think it's a problem with the new VS2010.
Should i try with some other software for profiling (better if free).
#11
.... i just tried the AMD profiler, and he simply left me speechless. I could not find a better software, and especially for this price (free).
A screenshot:

The Official WebPage: AMD CodeAnalyst Performance Analyzer
03/24/2012 (10:45 pm)
So wowww ....
.... i just tried the AMD profiler, and he simply left me speechless. I could not find a better software, and especially for this price (free).
A screenshot:

The Official WebPage: AMD CodeAnalyst Performance Analyzer
#12
honestly though, there is some C++ work that we'd like to do that has nothing to do with optimizing CURRENT code. sometimes, its better to start looking ahead.
03/24/2012 (10:54 pm)
@Alfio - ty much! We'll look into this and see what we can find out :)honestly though, there is some C++ work that we'd like to do that has nothing to do with optimizing CURRENT code. sometimes, its better to start looking ahead.
#13
03/24/2012 (11:04 pm)
@Paul, you mean to change engine, or rewrite part of the T3D?
#14
03/24/2012 (11:08 pm)
@Alfio - rewrite part of T3D
#15
Can't remember their username now ... :\
03/25/2012 (6:38 am)
Didn't someone do this some time ago but either not finish, release or moved on? I definitely remember them reporting a gazillion errors ....Can't remember their username now ... :\
#16
This is just the tip of the iceberg and I think some of the things Konrad pointed out can also be addressed in iT2D. So, thanks for the blog =)
03/25/2012 (8:32 am)
Excellent blog Konrad. Also, great replies from everyone. Even the smaller things stack up. For example, I've spent more time in the source code for iTorque 2D than any previous engine. Why? Because tuning a game for a mobile device is unavoidable. One of the most immediate and troublesome issues is the typecasting (U32 vs S32). We've always let the warning count slip and accepted that, but now it has bit us in the ass. iTorque 2D has thousands (oh dear lord) of warnings because of the typecasting. In addition to that being a red flag, it makes it impossible to spot new or worse issues. Joe spent some time cleaning up the source code for iT2D and got he warning count waaaaaay down. The work isn't finished, but it's not far from it.This is just the tip of the iceberg and I think some of the things Konrad pointed out can also be addressed in iT2D. So, thanks for the blog =)
#17
03/25/2012 (9:16 am)
Thanks Mich, great to see such response!
#18
03/29/2012 (9:42 am)
I think this is an excellent move. You never know when one of these little issues can turn into something quite big (like a crash). Liking what I see here.
#19
03/29/2012 (7:52 pm)
@Alfio - http://www.garagegames.com/community/blogs/view/21617 is what im talking about with the T3D re-write
#20
04/05/2012 (4:49 pm)
After correcting a good number of "coding errors", i measured a performance increase of more than 30%. I moved from 24fps to 32fps
Torque Owner Alfio Saitta
Collateral Studios
With the fixs u can see the results also visually.
Edit: Perhaps a next step should be to clean up the code and fix some bugs. Maybe for a version 1.2.1 ?!?