Writing the perfect code
by Steve D · in Torque Game Engine · 04/18/2008 (7:21 am) · 13 replies
I'm curious if anyone else falls into this "trap" - You are coding a feature for your game which includes multiple functions, etc. Just when you get it done and it's working you start to think the code is pretty sloppy and not very expandable for future changes so you go back, break it, and re-do it. Now you think it's solid and you can mark it as done but it keeps bothering you because you are sure there is a "better way" of writing it and go back and change it some more. Then you realize just how much time you have invested and it kind of almost feels like you are making yourself go in circles in pursuit of writing code that is perfect from all view points.
Anyone else do this or am I just a compulsive coder that wastes too much time on the same feature?
Anyone else do this or am I just a compulsive coder that wastes too much time on the same feature?
#2
I actually tend to fall into this quite a bit when writing behaviors in TGB. I like my behaviors to be general enough that they'll be usable in future projects. That can cause me to over think them sometimes.
04/18/2008 (8:08 am)
I've fallen into that a few times, mostly when it's code I'm planning on reusing. Mostly that's due more to trying to package it up in a nice little box that will fit as generally as possible in as many situations as possible. That's when I realize that I've written overly complex code for simple tasks.I actually tend to fall into this quite a bit when writing behaviors in TGB. I like my behaviors to be general enough that they'll be usable in future projects. That can cause me to over think them sometimes.
#3
This is really only useful if you're solo, and even then, you'll come back to it after a while and if it's not making sense then it'll possibly take you more time to understand it again than it would have taken to comment and structure it in the first place.
04/18/2008 (8:10 am)
Quote:
If its just a once off (which can be rare) who cares, if it works how you like and performs fast enough to be acceptable...
This is really only useful if you're solo, and even then, you'll come back to it after a while and if it's not making sense then it'll possibly take you more time to understand it again than it would have taken to comment and structure it in the first place.
#4
To reinforce Ashley's point, your original statement:
Implies that you think it is bad to do what you describe--properly managed it's not bad at all, in fact it's an excellent way to mature code.
Managers of projects run into this especially--they (understandably so) have serious issues with understanding the value of learning as part of development, and when a developer mentions the words "throw out and do again", they cringe, because it sounds like waste to them.
It's not (again, if done for the right reasons)--and knowing when to throw out a prototype and start from scratch (and of course factoring in all of the lessons you learned while making the prototype) is one of the smartest things you can do.
04/18/2008 (8:17 am)
Quote:
If you plan to reuse it, then the most common method is to prototype first, throw away then build the real one.
To reinforce Ashley's point, your original statement:
Quote:
I'm curious if anyone else falls into this "trap" - You are coding a feature for your game which includes multiple functions, etc. Just when you get it done and it's working you start to think the code is pretty sloppy and not very expandable for future changes so you go back, break it, and re-do it.
Implies that you think it is bad to do what you describe--properly managed it's not bad at all, in fact it's an excellent way to mature code.
Managers of projects run into this especially--they (understandably so) have serious issues with understanding the value of learning as part of development, and when a developer mentions the words "throw out and do again", they cringe, because it sounds like waste to them.
It's not (again, if done for the right reasons)--and knowing when to throw out a prototype and start from scratch (and of course factoring in all of the lessons you learned while making the prototype) is one of the smartest things you can do.
#5
04/18/2008 (8:59 am)
To steal from another genre, "Good code is never finished, only abandoned."
#6
I never do this. I plan at the outset to use all the code I start writing. I've found that if I do this, it's a lot easier to be in the mindset of writing code that asserts all the preconditions it needs to in order to be functional, and log anything that's abnormal. It's much easier to relax preconditions and handle a previously unhandled case than it is to try to handle all possible cases and conditions at the outset (without even knowing if they'll ever come up).
I never try to deliver a library of functionality without knowing what's really going to be used. I typically do top-down development, trying to drive code reuse from figuring out where some code I'm going to need can get reused later, and write and refactor it accordingly. I think trying to do design work from the bottom-up so that you can write very general purpose reusable code is very hard. Oftentimes it requires about 10% of the time to write code that handles 90% of the use case, and just asserts when presented with edge conditions it's not equipped to handle.
A very large part of my coding cycle involves refactoring, which typically is a process of migrating common functionality into a place where existing code in serveral places can make use of it. Also, in the case of Torque, I've spent a lot of time in the engine migrating publicly accessible things into private or protected. If you do this first, it becomes apparent which parts of the engine you can replace in a modular way and what you'll need to patch up to have it all still work.
In any case, I'd love to see some before and after examples, Steve -- maybe you're talking about refactoring and not reimplementation, really. I feel like I used to do this a lot when I first started coding, and I rarely recode implementations now (unless I'm doing it for performance reasons, or there is some glaring flaw in the design).
04/18/2008 (11:23 am)
Quote:
If you plan to reuse it, then the most common method is to prototype first, throw away then build the real one.
I never do this. I plan at the outset to use all the code I start writing. I've found that if I do this, it's a lot easier to be in the mindset of writing code that asserts all the preconditions it needs to in order to be functional, and log anything that's abnormal. It's much easier to relax preconditions and handle a previously unhandled case than it is to try to handle all possible cases and conditions at the outset (without even knowing if they'll ever come up).
I never try to deliver a library of functionality without knowing what's really going to be used. I typically do top-down development, trying to drive code reuse from figuring out where some code I'm going to need can get reused later, and write and refactor it accordingly. I think trying to do design work from the bottom-up so that you can write very general purpose reusable code is very hard. Oftentimes it requires about 10% of the time to write code that handles 90% of the use case, and just asserts when presented with edge conditions it's not equipped to handle.
A very large part of my coding cycle involves refactoring, which typically is a process of migrating common functionality into a place where existing code in serveral places can make use of it. Also, in the case of Torque, I've spent a lot of time in the engine migrating publicly accessible things into private or protected. If you do this first, it becomes apparent which parts of the engine you can replace in a modular way and what you'll need to patch up to have it all still work.
In any case, I'd love to see some before and after examples, Steve -- maybe you're talking about refactoring and not reimplementation, really. I feel like I used to do this a lot when I first started coding, and I rarely recode implementations now (unless I'm doing it for performance reasons, or there is some glaring flaw in the design).
#7
To try to give a semi-hypothetical example, let's say that I want to make a connect-4 game in TGB. I'll probably prototype this with an array as my underlying storage structure, and build things around what you can do with arrays--manipulate columns, rows, and move objects from one grid coordinate to another.
I may discover however along the way to getting a "first look" implementation that (for whatever reason), an array limits my flexibility in game mechanics--maybe I want to have special scrabble-style row/column modifiers that are temporary in nature, and are "outside" of my array bounds...or maybe I come up with some cool way of the game map dynamically changing in size based on game state changes.
If I operate under the mindset of "I use an array, and don't want to waste code, so I have to continue using an array no matter what changes", I may find myself creating all sorts of special checks, unusual edge cases, and other clumsy code because I continue to attempt to force an array to do things that I want it to do, instead of what it's designed to do--all because both my knowledge of the game, and my knowledge of what arrays can and cannot do have evolved.
In my experience (and several others!), the smartest thing to do would be to take the lessons I learned from my initial prototype, but throw away the actual implementation completely, and "start from scratch"--the trick is however, that I'm not starting from scratch--I'm starting from an evolved position of understanding of both how my game has evolved, as well as awareness of limitations of arrays, and other programming decisions I made in the first prototype. Instead of thousands of lines of clunky code that handles all sorts of weird conditions that only exist in my original prototype, I can re-design, and re-implement, a much cleaner solution--which in the big picture of my project, will increase not only my efficiency, but in many ways help to solve problems later on down the road.
04/18/2008 (11:36 am)
The main principle I'm trying to get at is that it is extremely rare for a developer to know exactly what is required, and therefore also extremely rare to know exactly the best way to implement things. This is especially important when looking at developing a game from an idea, since the game is an ever-evolving implementation, where many times design and development go hand in hand, and iterate off of each other.To try to give a semi-hypothetical example, let's say that I want to make a connect-4 game in TGB. I'll probably prototype this with an array as my underlying storage structure, and build things around what you can do with arrays--manipulate columns, rows, and move objects from one grid coordinate to another.
I may discover however along the way to getting a "first look" implementation that (for whatever reason), an array limits my flexibility in game mechanics--maybe I want to have special scrabble-style row/column modifiers that are temporary in nature, and are "outside" of my array bounds...or maybe I come up with some cool way of the game map dynamically changing in size based on game state changes.
If I operate under the mindset of "I use an array, and don't want to waste code, so I have to continue using an array no matter what changes", I may find myself creating all sorts of special checks, unusual edge cases, and other clumsy code because I continue to attempt to force an array to do things that I want it to do, instead of what it's designed to do--all because both my knowledge of the game, and my knowledge of what arrays can and cannot do have evolved.
In my experience (and several others!), the smartest thing to do would be to take the lessons I learned from my initial prototype, but throw away the actual implementation completely, and "start from scratch"--the trick is however, that I'm not starting from scratch--I'm starting from an evolved position of understanding of both how my game has evolved, as well as awareness of limitations of arrays, and other programming decisions I made in the first prototype. Instead of thousands of lines of clunky code that handles all sorts of weird conditions that only exist in my original prototype, I can re-design, and re-implement, a much cleaner solution--which in the big picture of my project, will increase not only my efficiency, but in many ways help to solve problems later on down the road.
#8
One point I was going to make in my post was that choosing appropriate data structures at the outset gives you a substantial leg up, in terms of not reworking your code. I didn't actually hit on that point after all.
If I find myself writing a "find" function using some property of an object as it's key, operating over a vector, array, or list of objects, and I see that I'll be using that find function prolificly, I typically will rethink my choice of data structure, and perhaps replace with a hash table or some other data structure that allows more efficient retrieval. I think it also helps to have spent a lot of time in other languages that have environments with runtimes and SDKs with nice collection APIs.
I think picking correct data structures at the outset helps immensely... being able to replace them when necessary is also hugely important, which is why a generalized, well-debugged set of low level data structures is nice to use. Unlike, say, rolling another ad-hoc hash table implementation in the middle of a class implementation -- and frankly, this is my one big gripe with Torque... there's little encapsulation of functionality and loads of ah-hoc linked list and hash table implementations embedded and intermixed with the classes themselves, where it doesn't belong... Namespace, Dictionary, NetEventNote, Convex, etc. all contain examples of this. I suppose one could argue that there was a time when doing so was the most expedient thing to do code-wise, and produced faster code than using another API to encapsulate that functionality, which would have ostensibly introduced function call overhead, at the very least -- I don't think there's any performance argument there anymore, however.
I think if you're writing code where you decide to add a member to a class that is the "next" pointer for a singley linked list implementation for this class, you should really take one step back and think hard about that.
That implementation is entirely ad-hoc, error-prone, hard to change to something else when necessary (a hash table or maybe a vector), and, allows you to only place objects of this class in one list at a time using that particular member.
I found STL to be unwieldy before, but now that I can actually see container contents very easily (in Visual Studio 2005), I tend to use it a lot.
04/18/2008 (1:15 pm)
Excellent points Stephen, and I'm glad you gave that kind of an example.One point I was going to make in my post was that choosing appropriate data structures at the outset gives you a substantial leg up, in terms of not reworking your code. I didn't actually hit on that point after all.
If I find myself writing a "find" function using some property of an object as it's key, operating over a vector, array, or list of objects, and I see that I'll be using that find function prolificly, I typically will rethink my choice of data structure, and perhaps replace with a hash table or some other data structure that allows more efficient retrieval. I think it also helps to have spent a lot of time in other languages that have environments with runtimes and SDKs with nice collection APIs.
I think picking correct data structures at the outset helps immensely... being able to replace them when necessary is also hugely important, which is why a generalized, well-debugged set of low level data structures is nice to use. Unlike, say, rolling another ad-hoc hash table implementation in the middle of a class implementation -- and frankly, this is my one big gripe with Torque... there's little encapsulation of functionality and loads of ah-hoc linked list and hash table implementations embedded and intermixed with the classes themselves, where it doesn't belong... Namespace, Dictionary, NetEventNote, Convex, etc. all contain examples of this. I suppose one could argue that there was a time when doing so was the most expedient thing to do code-wise, and produced faster code than using another API to encapsulate that functionality, which would have ostensibly introduced function call overhead, at the very least -- I don't think there's any performance argument there anymore, however.
I think if you're writing code where you decide to add a member to a class that is the "next" pointer for a singley linked list implementation for this class, you should really take one step back and think hard about that.
That implementation is entirely ad-hoc, error-prone, hard to change to something else when necessary (a hash table or maybe a vector), and, allows you to only place objects of this class in one list at a time using that particular member.
I found STL to be unwieldy before, but now that I can actually see container contents very easily (in Visual Studio 2005), I tend to use it a lot.
#9
As you demonstrate, it's critical to make smart decisions when you have the information necessary to make those decisions, and use the resources that are available to you. Torque is a prime example in some ways of what you mention--in many cases stand-alone implementations of basic (and even complex) data structures are scattered all about, when in a perfect code world, they shouldn't be. It's a lesson learned for sure, and one of the supreme advantages over in the engine R&D department is that they aren't under extreme deadline pressures to "get 'er done" and out the door, so they have time to do the "right thing" when it comes to using existing classes, and creating new ones when appropriate.
I also think in some ways we're approaching this from two perspectives:
--you know exactly what has to be done, it's not an experiment or learning process, and you have a full (or mostly complete) design that you simply need to implement. In this case, I agree with all of your points completely--use the right structures, at the right times, in the right ways. Prototyping and iterative development is not an excuse to be sloppy, it's a strategy to be flexible.
--you are learning, or experimenting, or simply playing around to generate new project ideas. This is where prototyping and iterative development shines, because you are intentionally exploring. It's just critical to not paint yourself into a corner, and refuse to "give up" on that particular iteration and move back to a previous one.
I think an article from Gamasutra entitled Future Play 2007: Keeping The Pace With Industry Innovation really covers the topic very well (at an introductory level), and the concept of "Learning to Fail" is a critical one, especially for small and independent studios.
04/18/2008 (1:42 pm)
I think the key here is properly managed.As you demonstrate, it's critical to make smart decisions when you have the information necessary to make those decisions, and use the resources that are available to you. Torque is a prime example in some ways of what you mention--in many cases stand-alone implementations of basic (and even complex) data structures are scattered all about, when in a perfect code world, they shouldn't be. It's a lesson learned for sure, and one of the supreme advantages over in the engine R&D department is that they aren't under extreme deadline pressures to "get 'er done" and out the door, so they have time to do the "right thing" when it comes to using existing classes, and creating new ones when appropriate.
I also think in some ways we're approaching this from two perspectives:
--you know exactly what has to be done, it's not an experiment or learning process, and you have a full (or mostly complete) design that you simply need to implement. In this case, I agree with all of your points completely--use the right structures, at the right times, in the right ways. Prototyping and iterative development is not an excuse to be sloppy, it's a strategy to be flexible.
--you are learning, or experimenting, or simply playing around to generate new project ideas. This is where prototyping and iterative development shines, because you are intentionally exploring. It's just critical to not paint yourself into a corner, and refuse to "give up" on that particular iteration and move back to a previous one.
I think an article from Gamasutra entitled Future Play 2007: Keeping The Pace With Industry Innovation really covers the topic very well (at an introductory level), and the concept of "Learning to Fail" is a critical one, especially for small and independent studios.
#10
And I'd have to say, my feelings on this are summarized exceptionally well by Joel Spolsky (of Joel on Software fame).
I'll just give you the synopsis:
Granted, you're talking about throwing away a prototype that hasn't yet seen the light of day. And this is talking about debugged and shipped software. But, the crux of the article is that if you've worked with a codebase, you understand something about it... you have a knowledgebase, and you understand its shortcomings. If you throw it all away, and start from scratch, sure you're gonna put a better architecture or design in place that is informed by all the new knowledge you have about what you should have done. but you're not going to create fewer bugs per line of new code than existed in the old code (programmers rarely do). You're more likely to have many more, because the old code, as ugly as it is, has been debugged (even if it's a prototype... I'm assuming it's at least a working prototype... some amount of testing and debugging went in to that).
I always think it's better to iteratively adapt the code, changing and refactoring it piecemeal as you can, rather than throwing it all away wholesale. In the end, it might even look entirely different, but at least you've had something that functions (more or less) throughout the entire process. If you throw it all away and start at scratch, you have nothing until your new version is close to complete. And even then, once you're mostly done, you don't know much about how the code behaves. You might have tested it in some scenarios, but you don't know where all the bugs are.
This is the approach we've tried to take with the Torque engine itself, privatizing and encapsulating as we went, so that we could then take things like the Resource Manager, the interpreter, or whatever, and replace its guts and still have a functioning engine the entire time.
I can't tell you how many times I've thrown away a refactoring because I would have introduced many more bugs checking it into the repository than existed in the old ugly, hard-to-read, hard-to-maintain, or just-plain-inefficient code that I was trying to "fix".
04/18/2008 (4:08 pm)
Well, I'll just go back to this point:Quote:
In my experience (and several others!), the smartest thing to do would be to take the lessons I learned from my initial prototype, but throw away the actual implementation completely, and "start from scratch"--the trick is however, that I'm not starting from scratch--I'm starting from an evolved position of understanding of both how my game has evolved, as well as awareness of limitations of arrays, and other programming decisions I made in the first prototype.
And I'd have to say, my feelings on this are summarized exceptionally well by Joel Spolsky (of Joel on Software fame).
I'll just give you the synopsis:
Quote:
They did it by making the single worst strategic mistake that any software company can make:
They decided to rewrite the code from scratch.
Granted, you're talking about throwing away a prototype that hasn't yet seen the light of day. And this is talking about debugged and shipped software. But, the crux of the article is that if you've worked with a codebase, you understand something about it... you have a knowledgebase, and you understand its shortcomings. If you throw it all away, and start from scratch, sure you're gonna put a better architecture or design in place that is informed by all the new knowledge you have about what you should have done. but you're not going to create fewer bugs per line of new code than existed in the old code (programmers rarely do). You're more likely to have many more, because the old code, as ugly as it is, has been debugged (even if it's a prototype... I'm assuming it's at least a working prototype... some amount of testing and debugging went in to that).
I always think it's better to iteratively adapt the code, changing and refactoring it piecemeal as you can, rather than throwing it all away wholesale. In the end, it might even look entirely different, but at least you've had something that functions (more or less) throughout the entire process. If you throw it all away and start at scratch, you have nothing until your new version is close to complete. And even then, once you're mostly done, you don't know much about how the code behaves. You might have tested it in some scenarios, but you don't know where all the bugs are.
This is the approach we've tried to take with the Torque engine itself, privatizing and encapsulating as we went, so that we could then take things like the Resource Manager, the interpreter, or whatever, and replace its guts and still have a functioning engine the entire time.
I can't tell you how many times I've thrown away a refactoring because I would have introduced many more bugs checking it into the repository than existed in the old ugly, hard-to-read, hard-to-maintain, or just-plain-inefficient code that I was trying to "fix".
#11
Even programs that have been released by major companies have flaws.
For example, Morrowind III - the Eder Scrolls. They decided not do do somethibg with a character, but they never changed the code. When you killed this unimportant character, it said something to the effect of "You've ruined the world" even though you didn't.
Point number two... A novelist will read and reread and rewrite paragraphs over and over in an attempt to get it perfect. Years can go by and they keep rewriting.
My points are... there is no such thing as perfect. Everything can be improved and streamlined and be made more efficient. If it works, its best to release it and make improvements in the next project.
Make it work, then move on.
IMHO,
Tony
06/02/2008 (6:02 pm)
Just a couple quick points...Even programs that have been released by major companies have flaws.
For example, Morrowind III - the Eder Scrolls. They decided not do do somethibg with a character, but they never changed the code. When you killed this unimportant character, it said something to the effect of "You've ruined the world" even though you didn't.
Point number two... A novelist will read and reread and rewrite paragraphs over and over in an attempt to get it perfect. Years can go by and they keep rewriting.
My points are... there is no such thing as perfect. Everything can be improved and streamlined and be made more efficient. If it works, its best to release it and make improvements in the next project.
Make it work, then move on.
IMHO,
Tony
#12
Even programs that have been released by major companies have flaws.
For example, Morrowind III - the Eder Scrolls. They decided not do do somethibg with a character, but they never changed the code. When you killed this unimportant character, it said something to the effect of "You've ruined the world" even though you didn't.
Point number two... A novelist will read and reread and rewrite paragraphs over and over in an attempt to get it perfect. Years can go by and they keep rewriting.
My points are... there is no such thing as perfect. Everything can be improved and streamlined and be made more efficient. If it works, its best to release it and make improvements in the next project.
Make it work, then move on.
IMHO,
Tony
06/03/2008 (3:41 am)
Just a couple quick points...Even programs that have been released by major companies have flaws.
For example, Morrowind III - the Eder Scrolls. They decided not do do somethibg with a character, but they never changed the code. When you killed this unimportant character, it said something to the effect of "You've ruined the world" even though you didn't.
Point number two... A novelist will read and reread and rewrite paragraphs over and over in an attempt to get it perfect. Years can go by and they keep rewriting.
My points are... there is no such thing as perfect. Everything can be improved and streamlined and be made more efficient. If it works, its best to release it and make improvements in the next project.
Make it work, then move on.
IMHO,
Tony
#13
Even programs that have been released by major companies have flaws.
For example, Morrowind III - the Eder Scrolls. They decided not do do somethibg with a character, but they never changed the code. When you killed this unimportant character, it said something to the effect of "You've ruined the world" even though you didn't.
Point number two... A novelist will read and reread and rewrite paragraphs over and over in an attempt to get it perfect. Years can go by and they keep rewriting.
My points are... there is no such thing as perfect. Everything can be improved and streamlined and be made more efficient. If it works, its best to release it and make improvements in the next project.
Make it work, then move on.
IMHO,
Tony
06/03/2008 (3:49 am)
Just a couple quick points...Even programs that have been released by major companies have flaws.
For example, Morrowind III - the Eder Scrolls. They decided not do do somethibg with a character, but they never changed the code. When you killed this unimportant character, it said something to the effect of "You've ruined the world" even though you didn't.
Point number two... A novelist will read and reread and rewrite paragraphs over and over in an attempt to get it perfect. Years can go by and they keep rewriting.
My points are... there is no such thing as perfect. Everything can be improved and streamlined and be made more efficient. If it works, its best to release it and make improvements in the next project.
Make it work, then move on.
IMHO,
Tony
Torque 3D Owner Ashley Leach
If its just a once off (which can be rare) who cares, if it works how you like and performs fast enough to be acceptable... Once you get building you'll realise you don't have enough time to make everything perfect, and it certainly isn't cost effective.
Just my thoughts, but i wrote my thesis on cost estimation, its hard enough to work out how long building a feature takes without redoing working code :)
Cheers,
Ash