Improvement Ideas
by Mike Lilligreen · in Torque 2D Professional · 08/01/2013 (5:27 am) · 8 replies
I was curious and had a look at some of the things happening on the T3D side, and a couple things caught my eye.
1. TorqueScript Improvements
There's a pull request from James Urquhart that improves the console function calls and results in a nice bump in performance. Since some of us have to rely solely on scripting for all game logic, plus the iOS and eventual Android hardware specs that need to be targeted, this looks really tempting. Having a look at the files changed, I get the impression that T2D and T3D share pretty much the same Torquescript to C++ bridge. Could this potentially be something that we can almost copy and paste over to our code or are there some "gotchas" that might make this more difficult to port? James also has a branch in his repo that goes even further into improving Torquescript, but I don't know how complete/incomplete that is.
2. Entity Component System
Jeff Raab is working on a behavior/component system inspired in part by our behavior system for scripts. I love using behaviors so I found the topic interesting. His system goes a bit further though and it looks to be changing game entities to be more component based. While working on the documentation, I am getting a better understanding of the long inheritance chains found on a lot of the typical T2D classes (Sprite->SpriteBase->SceneObject->BehaviorComponent->DynamicConsoleMethodComponent->SimObject, as an example that was off the top of my head, perhaps I missed something). Would there be any benefit for T2D to ever go down the component route? Or is everything lightweight enough that there wouldn't necessarily be any sort of performance or ease of code/debugging benefit? I bring this up more for my knowledge and as a discussion topic, and not as a suggestion for change.
1. TorqueScript Improvements
There's a pull request from James Urquhart that improves the console function calls and results in a nice bump in performance. Since some of us have to rely solely on scripting for all game logic, plus the iOS and eventual Android hardware specs that need to be targeted, this looks really tempting. Having a look at the files changed, I get the impression that T2D and T3D share pretty much the same Torquescript to C++ bridge. Could this potentially be something that we can almost copy and paste over to our code or are there some "gotchas" that might make this more difficult to port? James also has a branch in his repo that goes even further into improving Torquescript, but I don't know how complete/incomplete that is.
2. Entity Component System
Jeff Raab is working on a behavior/component system inspired in part by our behavior system for scripts. I love using behaviors so I found the topic interesting. His system goes a bit further though and it looks to be changing game entities to be more component based. While working on the documentation, I am getting a better understanding of the long inheritance chains found on a lot of the typical T2D classes (Sprite->SpriteBase->SceneObject->BehaviorComponent->DynamicConsoleMethodComponent->SimObject, as an example that was off the top of my head, perhaps I missed something). Would there be any benefit for T2D to ever go down the component route? Or is everything lightweight enough that there wouldn't necessarily be any sort of performance or ease of code/debugging benefit? I bring this up more for my knowledge and as a discussion topic, and not as a suggestion for change.
About the author
#2
08/01/2013 (6:34 am)
I'm going to second that - we could look into the improvements made to T3D's TS interpreter. 
#3
I know there was some interest from the T2D folks, but I lost interest and have been focusing on T3D this last year. Honestly, the amount of work to connect to T2D is mostly done because both engines are so similar. The biggest issues are hook points as there is no separation of init and mainloop in the engine, and turning T2D into a DLL/PYD. So, if people really want it I might be able to find some time to make it happen. Also, this would have no effect on existing TS so you are not losing anything. So if improvements are made to TS speed wise this would have minimal to no effect on a Python interface.
If this is too far off topic I will shut up now.
08/02/2013 (3:44 pm)
I know I have mentioned this before, but I have not moved forward on this at all. I have not really looked at the T2D internals since last year. What I am talking about is using Python with T2D. Now, I am not sure it is faster, but it is a much more supported and mature VM. It also has the ability to bridge to V8 which is runtime compiled Javascript and can do contexts to isolate code (potential for safe user side scripting). Python can also be runtime compiled via sub processes and using a JIT compiler called PyPy. PyPy is not as rich as Python itself, but this could be done for higher speed algorithms that could be pushed to a process and communicated with using thread semantics. Finally, you can also put code into OpenCL using PyOpenCL. For inherently parallel tasks you can't beat it. I could see AI, physics and other high demand tasks running on OpenCL using either a CPU driver or GPU driver. On top of all of this you would get access to everything else Python has to offer including some really good networking libraries.I know there was some interest from the T2D folks, but I lost interest and have been focusing on T3D this last year. Honestly, the amount of work to connect to T2D is mostly done because both engines are so similar. The biggest issues are hook points as there is no separation of init and mainloop in the engine, and turning T2D into a DLL/PYD. So, if people really want it I might be able to find some time to make it happen. Also, this would have no effect on existing TS so you are not losing anything. So if improvements are made to TS speed wise this would have minimal to no effect on a Python interface.
If this is too far off topic I will shut up now.
#4
You mentioned your Python implementation would have no effect on TS. Are you able to mix script files from the different languages? So have half the game scripts in TS and the other half in Python?
08/03/2013 (2:08 am)
Nah, it's not off topic. In general, I like the idea of T2D supporting additional scripting languages. It's one less potential barrier to getting people to adopt the engine. I've seen posts on general game dev sites where Torque is blown off with some response like "I don't want to learn Torquescript".You mentioned your Python implementation would have no effect on TS. Are you able to mix script files from the different languages? So have half the game scripts in TS and the other half in Python?
#5
08/03/2013 (2:08 am)
double post
#6
Sorry: double quote... ;)
That is actually how I am currently creating my game. A mixture of TS and Python. I intend to keep most of the game specific logic in Python. While my startup scripts are in TS for now.
Right now I am only supporting Python with my interface. If someone wants other languages they will have to write the additional code to support them.
Now, the way I have written the interface allows you to interact between Python and TS both ways. You can make TS call Python callbacks that look like TS functions, and call TS from Python. It will change how you organize your code a bit. I also don't know how hard it would be to make it work on other platforms as my focus has been on the PC with Windows. But Python is an option. I do think you can get Python on many mobile platforms, but I have not done this kind of work myself. So it may not be a good fit for those systems.
08/03/2013 (2:29 am)
Quote:Are you able to mix script files from the different languages? So have half the game scripts in TS and the other half in Python?Yes
Quote:Are you able to mix script files from the different languages? So have half the game scripts in TS and the other half in Python?Yes
Sorry: double quote... ;)
That is actually how I am currently creating my game. A mixture of TS and Python. I intend to keep most of the game specific logic in Python. While my startup scripts are in TS for now.
Right now I am only supporting Python with my interface. If someone wants other languages they will have to write the additional code to support them.
Now, the way I have written the interface allows you to interact between Python and TS both ways. You can make TS call Python callbacks that look like TS functions, and call TS from Python. It will change how you organize your code a bit. I also don't know how hard it would be to make it work on other platforms as my focus has been on the PC with Windows. But Python is an option. I do think you can get Python on many mobile platforms, but I have not done this kind of work myself. So it may not be a good fit for those systems.
Quote:"I don't want to learn Torquescript"Maybe I am simple, but I read this as "I don't want to do whatever it takes to make my game." So to me that is a glass half empty attitude. I understand people only have so much time, but TS is no harder than C++, Python, Basic, Pascal, Javascript, Java, Lua, etc, etc, etc... Once you learn one the rest seem simple.
#7
Sounds good. I was just thinking in terms of a community behavior sharing resource, where some people might write behaviors in TS, others in Python/some other language T2D supported, It would be good to have it be "plug and play", which you have covered.
The way you have it setup, the bridge is between Python and TS and not Python and C++? Sorry for all the questions, I'm just not familiar with the technical details on how a scripting language is bound to the core engine. I simply like how Melv set up the TAML formats, where you can have any combination of XML, JSON, or binary and the engine can simply import/export based on the file extension. That might be an apples to oranges comparison in relation to scripting languages though, not sure.
In any case, if you have the time to put something together that gets Python working in T2D, then go for it. I think to get it incorporated into the master branch would require it to work seamlessly between all platforms though. If you only have time to get it working on Windows, you could just put it out there as a separate branch and perhaps someone else in the community can work on it further.
08/03/2013 (4:11 am)
I'll try to keep this to a single post and quote. :)Quote:That is actually how I am currently creating my game. A mixture of TS and Python. I intend to keep most of the game specific logic in Python. While my startup scripts are in TS for now.
Sounds good. I was just thinking in terms of a community behavior sharing resource, where some people might write behaviors in TS, others in Python/some other language T2D supported, It would be good to have it be "plug and play", which you have covered.
The way you have it setup, the bridge is between Python and TS and not Python and C++? Sorry for all the questions, I'm just not familiar with the technical details on how a scripting language is bound to the core engine. I simply like how Melv set up the TAML formats, where you can have any combination of XML, JSON, or binary and the engine can simply import/export based on the file extension. That might be an apples to oranges comparison in relation to scripting languages though, not sure.
In any case, if you have the time to put something together that gets Python working in T2D, then go for it. I think to get it incorporated into the master branch would require it to work seamlessly between all platforms though. If you only have time to get it working on Windows, you could just put it out there as a separate branch and perhaps someone else in the community can work on it further.
#8
No problems Mike. The interface is written in a mixture of C and C++. Python is written in C under the hood. So it uses C calling semantics. Of course T3D is written in C++, but can call C functions just fine. The reason I say that Python is tied to TS is because it is tied into the console itself. Otherwise I would have to bind to all the relevant C++ calls to enable Python to interact with the engine. This is a LOT of work. Instead I talk to the console and can call any Torque Script command from Python.
There are other custom functions for talking to specific pieces of the engine. These are generally things like initializing the engine, calling the main loop tick function (so Python controls the main loop), shutting down the engine, registering callbacks so the engine or Torque Script can call Python functions, grabbing the window handle (though not needed I did this because it may be useful), and more. I also do some calling optimizations to reduce lookups both on the Python side and the Torque Script side using memoization. My approach is similar to the approach taken by the Web browser plugin. Basically the browser plugin bridges the console to the Javascript engine of the web browser.
My goals for the ScriptT3D code were choice and options. I wanted access to databases, but I did not want to maintain that interface. Now with Python I have probably a 100 different choices for database plugins. Want a built in web server? Just go grab Twisted and have it run in another thread or another process using the exact same interface semantics to talk to either one. Want to spread out AI processing into multiple processes to take advantage of multicore machines? Just spawn some Python processes running V8 and talk to them using thread semantics. Those processes will be managed by the operating system and use every core in your system. Need to do a nasty calculation and don't want the game to pause? Shove that code onto an OpenCL computing device like the GPU.
Okay, I am ranting. You probably get it by now that I really like ScriptT3D because of the libraries it brings to the engine.
08/03/2013 (2:50 pm)
@Mike,No problems Mike. The interface is written in a mixture of C and C++. Python is written in C under the hood. So it uses C calling semantics. Of course T3D is written in C++, but can call C functions just fine. The reason I say that Python is tied to TS is because it is tied into the console itself. Otherwise I would have to bind to all the relevant C++ calls to enable Python to interact with the engine. This is a LOT of work. Instead I talk to the console and can call any Torque Script command from Python.
There are other custom functions for talking to specific pieces of the engine. These are generally things like initializing the engine, calling the main loop tick function (so Python controls the main loop), shutting down the engine, registering callbacks so the engine or Torque Script can call Python functions, grabbing the window handle (though not needed I did this because it may be useful), and more. I also do some calling optimizations to reduce lookups both on the Python side and the Torque Script side using memoization. My approach is similar to the approach taken by the Web browser plugin. Basically the browser plugin bridges the console to the Javascript engine of the web browser.
My goals for the ScriptT3D code were choice and options. I wanted access to databases, but I did not want to maintain that interface. Now with Python I have probably a 100 different choices for database plugins. Want a built in web server? Just go grab Twisted and have it run in another thread or another process using the exact same interface semantics to talk to either one. Want to spread out AI processing into multiple processes to take advantage of multicore machines? Just spawn some Python processes running V8 and talk to them using thread semantics. Those processes will be managed by the operating system and use every core in your system. Need to do a nasty calculation and don't want the game to pause? Shove that code onto an OpenCL computing device like the GPU.
Okay, I am ranting. You probably get it by now that I really like ScriptT3D because of the libraries it brings to the engine.
Torque Owner practicing01
MourningDoveSoft