ScriptT3D: Rewrite Almost Done

This rewrite has been a really good idea. I have been able to speed up parts of the code and learned a lot about the meta programming with Python. There are some pitfalls with doing meta programming with Python. Things that I never imagined would matter. Python is great for making changes to its structure on the fly. You can literally assign a function to an existing object or a class of objects. If you want the function to be bound to the object you have to tell the object that this is a bound method. Basically, does it act like a method on the object, or just a function attached to the object. They are slightly different animals. Why would you care about meta programming?

Good question, perhaps you don't, but it can be used to memoize parts of your code. What the heck does "memoize" mean?! Yeah, I asked the same question and found this answer. Basically it is a meta programming technique to reduce the overhead of calling functions that produce the same output every time you give it a given input. Say you are calculating the output of a sine function. You give it a specific radian and it spits out a specific value. So to memoize that function you create a dynamic lookup table with values that are added each time the output is needed for a specific value. The next time you get that specific radian input then you can spit out the precalculated value from a previous iteration. So you save processing power and make your code faster.

So now you are an expert on memoize or you don't care. Either way I am moving on to explain why it is cool. Every time a function is looked up and executed in any scripting language it takes a certain amount of overhead to find the function, assemble the parameters, and make the call. This is cpu cycles that you are not getting much value from. So, since I wanted to make my Python interface as fast as possible I decided to implement some memoization of the call itself. The first time a console function is called through my Python to T3D interface the code will:

1. Lookup that function or method to make sure it actually exists.
2. Wrap the call to the function inside a C++ object designed to track and call the function.
3. Return a lambda (nameless function) to the calling Python code that automagically wraps parameters to the call and calls the function.
4. Assigns the lambda to the attribute of the global namespace or the namespace of a SimObject proxy class. This basically makes it so you never have to do any of these steps again.

So in essence I dynamically assign a lambda function streamlined to call the T3D function as fast as it can and avoiding as many lookups as possible. Also, I do some speedup in the C++ code that calls the T3D functions:

1. For global functions I lookup the namespace and save it to a pointer for reference.
2. For class methods I do the same, and keep track of the class pointer so that I don't have to convert an ID or ClassName to a class pointer for each call. For tracking the class pointer I inform the SimObject that I have a reference to the object. That way if the object gets deleted it will set my pointer to NULL. This makes it so I only have to check for a NULL pointer before each call.

What is interesting is that console calls cannot avoid these lookups. Now, I cannot avoid the lookup in my Python code to find the function to call, but I have severely limited the Python overhead and the console lookup overhead for each call. The tradeoff is every function I call from Python to T3D is going to need a C++ object to track the call itself. So there is a slight memory hit during the first call to the function. So, in addition to using the SWIG -builtin option to speed up the interface I was able to speed up calls to the engine by reducing the overhead of name lookups in both Python and T3D.

I finally decided to remove one feature. The ability to export a Python object into the T3D console. Basically I would create a SimObject based object and create methods that call all the Python methods of that object. In addition I could manipulate the Python attributes of that Python object. The reason is I am just not seeing how useful this would be. My goal is to move to running most of my code from Python. Now, if someone decides they need this feature I will put it back in. The code for it still exists and it would not be hard to turn that feature back on.

Some parts of the code that still need work are mainly related to simobjects and their attributes. I need to add the ability to manipulate arrays attached to simobjects. I have generic attributes working, but not arrays.

Now for a new feature that I really like. One of the things I can do with the Python interface is redirect the T3D console output to anything. This is actually a hook in T3D that allows you to setup a callback that is called every time console data is output. Each console message returns an integer representing level and the actual text of the message. The level indicates if the message was a: normal message, a warning, or an error. You normally see these messages in the console as: white, cyan, and red. These give you an easy visual to know what the reason is for that message. It also helps you pick out warnings and errors amongst the sea of white messages.

Console output coloring:

Now I chose to change the coloring a bit for my example. The cool thing is if you don't like the coloring you can change it:

# messages of console are colored here
# change to your hearts desire
def consoleConsumerCallback(level, data):
    if level == 0:
        print "{0}{1}{2}".format(cr_Fore.BLUE, cr_Style.BRIGHT, data)
    elif level == 1:
        print "{0}{1}{2}".format(cr_Fore.CYAN, cr_Style.BRIGHT, data)
    elif level == 2:
        print "{0}{1}{2}".format(cr_Fore.RED, cr_Style.BRIGHT, data)

engine.ExportConsumer(consoleConsumerCallback)

To do the coloring I am using a Python library called colorama. It supports changing color to text consoles such as the Win32 console, Linux consoles, and the MacOSX console. So it is an inherently cross platform feature. I know this is a simplistic feature, but it is really useful when debugging code that gets output to the Windows console window. It also looks pretty and sometimes if you look at it right some colors look like they are floating above the other colors. So it can trip you out a bit. :)

That is all I have for now. This blog has code and a picture and lots and lots of text. So SOMETHING happened, just not sure what. ;)