Torque 3D's Achilles heel
by Konrad Kiss · in Torque 3D Professional · 03/24/2012 (11:49 am) · 184 replies
Recently I spent some time getting myself familiar with static code analysis and the more common (indie-budget freindly) tools associated with the topic. #AltDevBlogADay had several posts about such tools by Bruce Dawson and John Carmack. Definitely worth checking out.
So, it all began with me having a very elusive bug somewhere in code. I hoped that it would be something that a static code analyzer would pick up, so I lined up what I could from trial versions to open source software, and got some amazing results once these went through the Torque 3D 1.2 codebase.
I can't remember if the actual problem was found by these tools (it's been a while ago). What I recall is being taken aback by the results I got. But first a disclaimer...
I've been reviewing engine code changes for the past 4 years. Torque 3D is huge. One of the most surprising thing was the fact, that these code analyzers could - proportionally - find only a few problems. However, the few they did find were hard to see even when I knew the problem.
The kinds of problems I found primarily were:
- Uninitialized member variables used without initialization
- Checking the existence of an object and using the same object outside the check condition
- Passing local pointers that could become invalid
- Using && instead of &, || instead of | operations
- Bad casts for variables while the code assumes another cast (ie. U32 vs S32 - especially when negative values are used for error codes - common in some of the linked libraries)
- Typos, cut & paste programming mistakes that compile (probably the most vicious of all)
I'm only going to list my favorites, hoping that this will be a big enough motivation for everyone to start using some basic form of static code analysis. The complete list would be way too long for me to list here.
gfxD3D9TextureManager: using boolean evaluation against variables that were clearly intended to be bitmasks.
guiIconButtonCtrl.cpp: find the highlighted button, I dare you
processedFFMaterial.cpp: making sure (though what about blendDest?)
mRectF.cpp: intersectTriangle
winDInputDevice.cpp: POV_down, not much of a choice (twice in the same file)
rectClipper: clipRect (near eof)
win32Window.cpp: equilibrium
Finally - I need to clarify - by all means, no disrespect was intended towards Torque 3D developers. I would have done far worse most likely. I can't count the many new things in the source that made me go "Wow" and "Yey" - it is clearly visible that the team is bigger and better at what they do. Torque 1.2 is the best Torque yet - and I mean it by comparing the overall usability, the number of bugs, etc..
With this post I'm only trying to point at a solution that can save everyone from such blatant bugs in the future, while - perhaps - also nudging GG a little to take a step in the right direction. ;)
Cheers
--Konrad
So, it all began with me having a very elusive bug somewhere in code. I hoped that it would be something that a static code analyzer would pick up, so I lined up what I could from trial versions to open source software, and got some amazing results once these went through the Torque 3D 1.2 codebase.
I can't remember if the actual problem was found by these tools (it's been a while ago). What I recall is being taken aback by the results I got. But first a disclaimer...
I've been reviewing engine code changes for the past 4 years. Torque 3D is huge. One of the most surprising thing was the fact, that these code analyzers could - proportionally - find only a few problems. However, the few they did find were hard to see even when I knew the problem.
The kinds of problems I found primarily were:
- Uninitialized member variables used without initialization
- Checking the existence of an object and using the same object outside the check condition
- Passing local pointers that could become invalid
- Using && instead of &, || instead of | operations
- Bad casts for variables while the code assumes another cast (ie. U32 vs S32 - especially when negative values are used for error codes - common in some of the linked libraries)
- Typos, cut & paste programming mistakes that compile (probably the most vicious of all)
I'm only going to list my favorites, hoping that this will be a big enough motivation for everyone to start using some basic form of static code analysis. The complete list would be way too long for me to list here.
gfxD3D9TextureManager: using boolean evaluation against variables that were clearly intended to be bitmasks.
if (retTex->mProfile->isRenderTarget() && mslevel != 0 && (mDeviceCaps.Caps2 && D3DDEVCAPS2_CAN_STRETCHRECT_FROM_TEXTURES))
guiIconButtonCtrl.cpp: find the highlighted button, I dare you
ColorI fontColor = mActive ? (highlight ? mProfile->mFontColor : mProfile->mFontColor) : mProfile->mFontColorNA;
processedFFMaterial.cpp: making sure (though what about blendDest?)
result.blendSrc = GFXBlendOne;
result.blendSrc = GFXBlendOne;mRectF.cpp: intersectTriangle
if(contains(a) || contains(b) || contains(b))
winDInputDevice.cpp: POV_down, not much of a choice (twice in the same file)
newEvent.objInst = ( objInst == 0 ) ? SI_DPOV : SI_DPOV
rectClipper: clipRect (near eof)
out_rRect.extent.x = bottomR.x - out_rRect.point.x + 1; out_rRect.extent.x = bottomR.y - out_rRect.point.y + 1;
win32Window.cpp: equilibrium
newLeft = mClamp(newLeft, 0, newLeft);
newTop = mClamp(newLeft, 0, newTop);Finally - I need to clarify - by all means, no disrespect was intended towards Torque 3D developers. I would have done far worse most likely. I can't count the many new things in the source that made me go "Wow" and "Yey" - it is clearly visible that the team is bigger and better at what they do. Torque 1.2 is the best Torque yet - and I mean it by comparing the overall usability, the number of bugs, etc..
With this post I'm only trying to point at a solution that can save everyone from such blatant bugs in the future, while - perhaps - also nudging GG a little to take a step in the right direction. ;)
Cheers
--Konrad
About the author
http://about.me/konrad.kiss
#82
05/14/2012 (12:46 pm)
Let's does not so complicated. A simple svn server and access for all licensed users (after a request to a admin). An unique trunk and jobs assigned for open ticket. O_o
#83
Not to mention that if it's done with a one-account per person type of thing, that would end up being a hassle just creating/giving access to individual accounts.
05/14/2012 (1:13 pm)
I agree with Steve, I don't think there's too much risk involved with people giving out the credentials, mostly because there are easier ways of getting Torque if you were trying to share it around. Checking things out from BitBucket isn't nearly as casual as something like torrenting.Not to mention that if it's done with a one-account per person type of thing, that would end up being a hassle just creating/giving access to individual accounts.
#84
05/14/2012 (1:27 pm)
That is how TorqueX cev ended up going... one account per user, and Pino had a process he used to verify accounts, I think it was posting in one of the Private Forums in a specific thread.
#85
Alas, I haven't got much experience with gitHub specifically. I am familiar with other versioning tools though from work. Some of which have restricted access for legal reasons (which is why I brought some of the legal aspects up to begin with :) )
That aside, I am interested in both supporting this and (if possible) help out. But the latter depends on the free time I have available. Which is not always much, since I have a lot to do at work sometimes. I do want some family quality time too ;-P
I initially wrote a very long post with a lot of points around this. They basically circled around versions, dealing with users. Restrictions (we would have to use them, or we would circumvent GG licensing and that would be ... bad)
etc..
but I saved that on my computer, because I came to the conclusion that perhaps the best thing to do is this:
Let's do a small test. Let's have a limited amount of users (initially) that deal with bugs etc.. and other things. No more people allowed during this phase. And then we could think about solutions around all of this. And work things out before launching large scale access. I think this could be quite valuable. It could also help prove if this is at all something that could work out.
Lastly, bitbucket is quite cheap. Even the highest offer $80/month is pocket change really. Because sharing accounts irks me on so many levels I would have to say a definite no to that. One account per user is the way to go, that way we can also lock out a user should GG so request.
I also noted that Ogre is using bitbucket to store their mercurial repository, which speaks well for their service.
So I think it is an interesting alternative. Certainly cheaper than gitHub, and it seems more flexible.
05/14/2012 (1:44 pm)
@Konrad:Alas, I haven't got much experience with gitHub specifically. I am familiar with other versioning tools though from work. Some of which have restricted access for legal reasons (which is why I brought some of the legal aspects up to begin with :) )
That aside, I am interested in both supporting this and (if possible) help out. But the latter depends on the free time I have available. Which is not always much, since I have a lot to do at work sometimes. I do want some family quality time too ;-P
I initially wrote a very long post with a lot of points around this. They basically circled around versions, dealing with users. Restrictions (we would have to use them, or we would circumvent GG licensing and that would be ... bad)
etc..
but I saved that on my computer, because I came to the conclusion that perhaps the best thing to do is this:
Let's do a small test. Let's have a limited amount of users (initially) that deal with bugs etc.. and other things. No more people allowed during this phase. And then we could think about solutions around all of this. And work things out before launching large scale access. I think this could be quite valuable. It could also help prove if this is at all something that could work out.
Lastly, bitbucket is quite cheap. Even the highest offer $80/month is pocket change really. Because sharing accounts irks me on so many levels I would have to say a definite no to that. One account per user is the way to go, that way we can also lock out a user should GG so request.
I also noted that Ogre is using bitbucket to store their mercurial repository, which speaks well for their service.
So I think it is an interesting alternative. Certainly cheaper than gitHub, and it seems more flexible.
#86
05/14/2012 (1:54 pm)
I personally don't see this progressing using paid hosting as sooner or later there won't be enough funds etc. and it will go down. The account issue becomes a moot point if the repo will only stay up for the first month.
#87
05/14/2012 (1:56 pm)
What about individuals that have been with T3D for awhile. I would like to be a part, plus I've spent well over a few thousand dollars. I got a license(s) for TGE, TGEA, and T3D. I purchased T3D at the $1000 mark so I think long term member(s) that would want to participate should have the choice as well. Just an idea!!!
#88
Depending on how many would be included in the initial test (Should Konrad think it is a good idea) we could be talking $10-$20 / month which is nothing. I can pay that if money is an issue.
@Kory
Well, jumping on the train would be now don't you think :) ?
05/14/2012 (2:02 pm)
@RossDepending on how many would be included in the initial test (Should Konrad think it is a good idea) we could be talking $10-$20 / month which is nothing. I can pay that if money is an issue.
@Kory
Well, jumping on the train would be now don't you think :) ?
#89
05/14/2012 (2:06 pm)
@Michael, I suppose that works for the test, but eventually (be it one month or 5 or 10) it won't be paid and it will go down. That's the real issue. Hell, for a test on BitBucket you don't even need to pay anything (private repos are free and unlimited size, but limited to 5 accounts...so if your test was 5 people you'd be fine).
#90
That is what the test is for, feeling if it is possible. It could turn out that only a limited amount of people are actually seriously interested in this.
However, it could still work out in a beneficial way if updates find their way into GG official code. So, I think it is certainly worthwhile to try and see.
05/14/2012 (2:10 pm)
@RossThat is what the test is for, feeling if it is possible. It could turn out that only a limited amount of people are actually seriously interested in this.
However, it could still work out in a beneficial way if updates find their way into GG official code. So, I think it is certainly worthwhile to try and see.
#91
At that time, we could decide which way to go; I say, setup the accounts, start charging the $10 or $15 per month; I can commit to that easily even $25 or $30. Because of how most of the other microsoft pricing works, it is likely if a user doesn't pay you can delete his account, and thus not get charged for it. at that point, it really allows us to use it as a service, you have to pay to pay, its not a matter of it going up or down, is more of a matter of having people willing to pay the fees.
05/14/2012 (2:22 pm)
Beating the otherside of the "dead horse", TFS on Azure is Free, probably at least until TFS11 RTMs later this year, then if it works like their (Microsoft's) other services, we would have at least 60 days to migrate off, and I expect they would introduce a "trial" for a period of time for people to try it. That would get us to close to the end of the year anyway.At that time, we could decide which way to go; I say, setup the accounts, start charging the $10 or $15 per month; I can commit to that easily even $25 or $30. Because of how most of the other microsoft pricing works, it is likely if a user doesn't pay you can delete his account, and thus not get charged for it. at that point, it really allows us to use it as a service, you have to pay to pay, its not a matter of it going up or down, is more of a matter of having people willing to pay the fees.
#92
I rented a vps and i have installed almost everything. I end up to configure the authentication system, and we could begin.
AllInOne: Versioning Service, Ticket, Roadmaps, Changelog and all required.... i think O_o
https://lab.collateral-studios.eu/trac
05/14/2012 (2:35 pm)
Ok, i started to create a platform. Who wants to follow.... follow me :DI rented a vps and i have installed almost everything. I end up to configure the authentication system, and we could begin.
AllInOne: Versioning Service, Ticket, Roadmaps, Changelog and all required.... i think O_o
https://lab.collateral-studios.eu/trac
#93
Personally I don't trust Microsoft to do a good job when it comes
to versioning. And searching on Google doesn't really inspire confidence
either. But hey, I could be wrong :)
05/14/2012 (2:36 pm)
@JohnPersonally I don't trust Microsoft to do a good job when it comes
to versioning. And searching on Google doesn't really inspire confidence
either. But hey, I could be wrong :)
#94
I'm interested. And if the bandwidth turns out to be limited, I can help out with that.
05/14/2012 (2:51 pm)
@AlfioI'm interested. And if the bandwidth turns out to be limited, I can help out with that.
#95
05/14/2012 (2:51 pm)
@Alfio: Awesome! How much is the vps? I'd like to help you pay for it.
#96
05/14/2012 (2:57 pm)
Cool deal, sounds like Alfio has jumped in and got things started! Let us know what you need from our end.
#97
Crap, I just started a repo at Xp-Dev and setup a Paypal donation link for those who want to contribute. My space was already paid for as I use it for my consulting business. Anyway, thanks for taking the bull by the horns. If you find that service to be lacking let me know and I can post the links for the repo I setup. For now I will ignore it.
05/14/2012 (3:13 pm)
@Alfio,Crap, I just started a repo at Xp-Dev and setup a Paypal donation link for those who want to contribute. My space was already paid for as I use it for my consulting business. Anyway, thanks for taking the bull by the horns. If you find that service to be lacking let me know and I can post the links for the repo I setup. For now I will ignore it.
#98
@Michael, what have you read? I have several customers (fortune 100) that use TFS daily and we have never had any versioning issues. Visual Source Safe had some issues... but that is for LAN-based shops, and frankly I don't know anyone that uses it anymore. I personally have about 200 gigs of source and resources in our TFS server and also have not experienced any issues relating to versioning or performance
05/14/2012 (3:18 pm)
Lol, and I had created a site on TFS Azure...@Michael, what have you read? I have several customers (fortune 100) that use TFS daily and we have never had any versioning issues. Visual Source Safe had some issues... but that is for LAN-based shops, and frankly I don't know anyone that uses it anymore. I personally have about 200 gigs of source and resources in our TFS server and also have not experienced any issues relating to versioning or performance
#99
I'd like to say I also fired up an AWS instance, but no.. I had some whiskey instead. :)
The problem with Azure services and Microsoft dev tools is that they are usually frowned upon by those who do not use them much. :) Azure is really nice.. But it's oh so expensive outside the free tier!
Edit: I had whiskey because I almost managed to launch our game today. I missed that, but I really would have hated to miss the party with it. I usually consume alcohol every 2 years, and I think it shows.. Gah, I'll shut up..
05/14/2012 (3:45 pm)
:) Best community evar.I'd like to say I also fired up an AWS instance, but no.. I had some whiskey instead. :)
The problem with Azure services and Microsoft dev tools is that they are usually frowned upon by those who do not use them much. :) Azure is really nice.. But it's oh so expensive outside the free tier!
Edit: I had whiskey because I almost managed to launch our game today. I missed that, but I really would have hated to miss the party with it. I usually consume alcohol every 2 years, and I think it shows.. Gah, I'll shut up..
#100
I am up for whereever anyone wants to head. I don't mind spending the effort to manage the user accounts and whatever branching we want to do, its really not that much work. The best part is, its free, at least for now... and that would give is a good indicator as to how much activity we are going to get.
TFS has all of the work items, bugs, user stories, etc; and all of the functionallity mentioned by @Alfio. But its better integrated inside visual studio (for obvious reasons).
If anyone wants to have a go at it with TFS, let me know.
05/14/2012 (3:52 pm)
@Konrad - TFS on Azure is just TFS over the internet with SQL AZURE and some tweaking of the Server to make it handle multiple instances better. All of those changes I expect will make it into the final product.I am up for whereever anyone wants to head. I don't mind spending the effort to manage the user accounts and whatever branching we want to do, its really not that much work. The best part is, its free, at least for now... and that would give is a good indicator as to how much activity we are going to get.
TFS has all of the work items, bugs, user stories, etc; and all of the functionallity mentioned by @Alfio. But its better integrated inside visual studio (for obvious reasons).
If anyone wants to have a go at it with TFS, let me know.
Associate Steve Acaster
[YorkshireRifles.com]
If someone is going to pass on their login/download info for this, then they've probably already done it for the GG original version by uploading it somewhere that they should not have.
I'd go for an honour system where you assume the licensees aren't going to give out the login/download info ... 'cos that's really what GG already have once the user has bought Torque, an honour system that they will not upload it to the net for a free-for-all.