Documenting the beast
by Daniel Buckmaster · in Torque 3D Professional · 06/16/2014 (3:47 am) · 8 replies
Documentation seems to always have been one of Torque's flaws despite valiant efforts to combat this. I've been wondering about how we could combat this in a practical way. I stumbled into a Reddit thread where someone was having difficulty using a Haskell library, and an experienced coder dropped in to help them out. They came to the library completely fresh, and documented their thought process as they went through trying to understand this person's problem and the library itself.
That evening, I spent a while stepping through the asset loading system to test Andrew's assimp import abilities. I'd never been into the ResourceManager code or shape loading before - it was a voyage of discovery trying to work out what each piece did. Maybe there was documentation I should have looked up first that would have told me that Resource<> overloads its assignment operator and that's where shape loading happens - but I had to figure it out the hard way.
That got me thinking - what if we were to take a similar approach to providing more engine documentation? We don't all need to be grandmasters with encyclopaedic knowledge of the engine's internals to be able to write a tutorial. As long as you're learning something, then you probably have something to share to help someone else learn that same thing. And it seems to me like most people in this community are constantly learning new things about how the engine works!
I've actually tried this approach once before - I wrote a non-expert's guide to WheeledVehicle datablock parameters. I just sat down for a day and played with all the numbers in the datablock to see what qualitative effects they had on the experience. People seemed to find it helpful.
So maybe we should think about this as a way to better document the engine. I think the advantage is anyone can do it - just write down what you're discovering as you do it! Sometime soon I'm going to take another dive into the resource manager and try writing one of these things to see if it's any good.
Opinions please.
EDIT: I wish I'd done this while I was paring down the Empty template for t3d-bones. That was a pretty epic week.
That evening, I spent a while stepping through the asset loading system to test Andrew's assimp import abilities. I'd never been into the ResourceManager code or shape loading before - it was a voyage of discovery trying to work out what each piece did. Maybe there was documentation I should have looked up first that would have told me that Resource<> overloads its assignment operator and that's where shape loading happens - but I had to figure it out the hard way.
That got me thinking - what if we were to take a similar approach to providing more engine documentation? We don't all need to be grandmasters with encyclopaedic knowledge of the engine's internals to be able to write a tutorial. As long as you're learning something, then you probably have something to share to help someone else learn that same thing. And it seems to me like most people in this community are constantly learning new things about how the engine works!
I've actually tried this approach once before - I wrote a non-expert's guide to WheeledVehicle datablock parameters. I just sat down for a day and played with all the numbers in the datablock to see what qualitative effects they had on the experience. People seemed to find it helpful.
So maybe we should think about this as a way to better document the engine. I think the advantage is anyone can do it - just write down what you're discovering as you do it! Sometime soon I'm going to take another dive into the resource manager and try writing one of these things to see if it's any good.
Opinions please.
EDIT: I wish I'd done this while I was paring down the Empty template for t3d-bones. That was a pretty epic week.
About the author
Studying mechatronic engineering and computer science at the University of Sydney. Game development is probably my most time-consuming hobby!
#2
1. IMO, fork and pull request the official wiki. We do need to overhaul the wiki a little bit to organise it better, but it's a nice visible location. As far as locations go.
2. I keep meaning to check those videos out :/. It does get lost, it's a massive problem. I think your thread's good as far as that goes, though I think there's just such a vast amount of information out there that it doesn't structure. For example, I'm never ever able to find the class hierarchy reference when I need it. And it has different information to the scripting reference manual, and TDN, and the 'official documentation'.
06/16/2014 (6:02 am)
Haha yes. To be honest I had a few people and engine areas in mind, but I feel like it might be a generally helpful way to look at creating more documentation.1. IMO, fork and pull request the official wiki. We do need to overhaul the wiki a little bit to organise it better, but it's a nice visible location. As far as locations go.
2. I keep meaning to check those videos out :/. It does get lost, it's a massive problem. I think your thread's good as far as that goes, though I think there's just such a vast amount of information out there that it doesn't structure. For example, I'm never ever able to find the class hierarchy reference when I need it. And it has different information to the scripting reference manual, and TDN, and the 'official documentation'.
#3
06/16/2014 (7:07 am)
YES! I had, for a while, been thinking there should be a greater effort to comment the engine's code to document what does what and how. It'd be great for new developers who want to make their own modifications. 
#4
06/16/2014 (10:47 am)
Best of luck. If you want sage-like advice, let me know.
#5
06/16/2014 (5:53 pm)
Lukas: just watched the first Nuny abusiness video, and yeah, it was basically exactly what I was thinking of, though I think it suffers a bit for being a video with lots of dead air. If it were written down you could get rid of those and make it a bit more focused and helpful, and the reader wouldn't have to wait for you to read the code (the ThreadPool rabbit hole was deep and dark...). More condensed. Anyway, that said the video was great and did teach me a lot. I'm going to set aside some time to go through the rest of the series!
#6
06/22/2014 (4:33 am)
I made a post exploring the unit test framework. See what you think. Is it actually helpful? Be warned: if you actually want to learn from it, it's probably a good idea to open up Visual Studio and follow along. I frequently jump around using 'find all references' and text searches. 
#7
I always thought that T3D 4.1 would be a milestone for documentation .... but still a long way off :(
We have to get used on comment the code, it is very frustrating to spend a few hours to understand how work a part of T3D ... to forget in a few months.
I'd rather have the code and the documentation together as much as possible, no chance of that is lost over time. But add some tutorials/manuals to a wiki can be very good.
07/01/2014 (1:30 pm)
@Daniel, great post :)I always thought that T3D 4.1 would be a milestone for documentation .... but still a long way off :(
We have to get used on comment the code, it is very frustrating to spend a few hours to understand how work a part of T3D ... to forget in a few months.
I'd rather have the code and the documentation together as much as possible, no chance of that is lost over time. But add some tutorials/manuals to a wiki can be very good.
#8
The search function of this site should be reviewed; there's so much documented over time waiting to be found again and again; but the ways to it are sometimes utterly time consuming...
07/07/2014 (7:25 pm)
If only we could make a dump out of Tom Spillman's brain and convert it into a markup language ;-)The search function of this site should be reviewed; there's so much documented over time waiting to be found again and again; but the ways to it are sometimes utterly time consuming...
Torque Owner Lukas Joergensen
WinterLeaf Entertainment
Anyways I think this would be great but issues:
1. Documenting is fun but cumbersome, it takes too long time and where do you do it? We need a documentation platform that makes it easy to make tutorials etc.
Maybe a WYSIWYG editor for BB codes that can be pasted into the resources section, or even a more advanced system that allows you to create beautiful (maybe even interactive?) tutorials in a simple manner. Does anyone know of services that does this?
2. Documentation kindda gets lost, I created the tutorial thread to combat this, but I believe the tutorial thread is just getting lost as well :P
You'll find that Nuny abusiness has done exactly what you are describing in his Engine source analysis, people just don't know about these source for information!
Some search engine for finding tutorials would be great, where you can search semantically as well as syntactically. I created a prototype product browser to solve this issue for add-ons and commercial products for T3D. (Though that product browser was just a prototype and a proof-of-concept I made, it's not active anymore as I've moved my website)