Community testing of TorqueScript optimisations

#21 Righty-ho. Think I've narrowed down the crashes to using multiple state timeouts in weaponImage.

useRemainderDT = true;//allow multiple state timeouts in a single tick

Haven't had a random lockup/crash since taking that out. Not sure if Debug Mode slowing the whole process down could be a reason why I couldn't get that particular issue to raise it's head outside of Release version or whether I'm now talking twaddle.

So apart from WTF is wrong with audio profiles, the sort order for some array functions, the crash when unmounting a weapon via it's FSM script which I reported in it's own thread and the bitStream Value Out Of Range when creating an object in script (like a dynamically scripted bullet - see my old resource on that) ---- everything else seems to be working fine. I've played a tonne of missions against my Ai, and halfway through a full campaign. Testing will continue, I'll restart a full new campaign with some script changes I've made to difficulty settings, and post here again if I notice anything dodgy.

#22 Okay .... after much faffing ... the issues I've described here only happen on "rebuilds" of the console refractor branch, whether GG Development or JamesU branch. They're not happening on the standard 2.0 MIT after rebuild of code.

I have tested:
JamesU consoleRefraction (old toolbox) - issues on rebuild
GG console refraction merge console development branch - issues on rebuild
GG standard 2.0 post rebuild - none of the issues I have listed in this thread.

Thus my conclusion is, console refraction has caused the issues listed in this thread ... whilst signifiactnly speeding up scripts performance.

[/science]

#23 @Steve:
Thanks so much for being the only community member to give these changes a thorough test. I'm working towards addressing the issues you've found and moving these changes to the development branch.

- Dave

#24 Glad to be of help, Dave.
:)

#25 Bumping this to remind everyone that we are still asking for more community feedback in testing James' script improvements. This has not been forgotten, but Dave and I both would like to resolve all possible issues before merging into the development branch.

#26 Had some issues with arrays early on from memory, however James resolved those. I don't have any metrics to share, but I might revisit this as there was some nice performance gains.

Also there was a a conflict with this resource, which I was using at the time.

#27 Once I get done with the stent I'm doing now on some of the code, I'll pull the changes and run it as stock, let you know if I find anything. I've just been very short of time in the last year because of work. Not intended but that's life.

#28 Okay, I finally pulled this branch down and merged the latest development head into it. In a script project based on the Empty template, I get a crash when loading a level in Release mode, but not in Debug. The last thing in the console is:

Connect request from: IP:0.0.0.0:0

I'm doing some more testing now - first thing is to use an unmodofied template project! I've also got a small script project (for Lukas's magazine) which is basically all-script so far, which should be a nice test of some of the nastier parts of TS's functionality (multidimensional arrays anyone?).

EDIT:

The Release-mode crash seems to exist in the templates. Steps I took:

git checkout consolefuncrefactor
git merge development

Compile engine (with recast in)
Copy .exe and .dll to Templates/Full/game
Run .exe
Play 'empty room' level

The crash happens after all the DAE preloading and so on, right when the level would usually load. Last entry in the console is the same as above.

FURTHER EDIT:

Same crash happens in my other project based on t3d-bones, but the console log is different (of course). I can get it to work with a debug executable, but on exit I get a popup with this message:

Quote:
There is a texture object leak, check the log for more details

And in the console:

Cur. D3DDevice ref count=1
GFXTextureObject Usage Report - 2 active TOs
---------------------------------------------------------------
 Addr   Dim. GFXTextureProfile  ProfilerPath DebugDescription
 87e1ed8  ( 800,  600)  PostFxTargetProfile    MainLoop -> CanvasRenderControls -> GameRenderWorld -> SceneGraph_renderScene -> SceneGraph_batchRenderImages -> Scene_renderObjects -> RenderPassManager_RenderPass -> RenderPassManager_Render    mBackBufferCopyTex
 87e1d38  ( 800,  600)  PostFxTargetProfile    MainLoop -> CanvasRenderControls -> GameRenderWorld -> SceneGraph_renderScene -> SceneGraph_batchRenderImages -> Scene_renderObjects -> RenderPassManager_RenderPass -> RenderPassManager_Render    PostEffect::_setupTarget
----- dump complete -------------------------------------------
c:devtorque3denginesourcegfxgfxdevice.cpp(275) : Fatal - There is a texture object leak, check the log for more details.

I assume this means my cleanup is wrong somehow? This doesn't happen with the stock engine, of course.

EDIT:

Okay, the texture error doesn't happen in the templates, or in regular t3d-bones. So I'm doing something funny in my custom cleanup that the engine doesn't catch normally. I think. The executables I was using were a bit old; I might double-check that without the console optimisations and check back. Until then, assume it's unrelated to these changes.

#29 I'm trying this branch out as well. Just did a successful build and testing out some stuff.

I'm also getting a "texture object leak" sort of error, but using only the development branch.

#30 Okay, I am just starting to do some testing on this. I have found one potential string error:

Mapping string: Visitor to index: 7
onServerMessage: 
  +- a: Welcome to a Torque application Visitor.

This string looks okay here, but in the console it has a bunch of weird characters in it. It tells me there is some string issues somewhere.

I am testing with a debug compile with nothing else in the engine except T3D 3.0. I am also using the empty template.

@James,
I don't know where to focus my efforts to help debug the console changes. My thinking is looking into the string error and possibly start writing some sort of unit testing of the console and possibly some of the C++ functions that interact with the console. Can you suggest some areas to look?

I did notice a significantly shorter startup time even in debug. This is a awesome!

#31 Ah, crap, I was testing against the wrong build... I will let you know what I find.

#32 @demolishun

Make sure when you use %s to print a console arg, cast it to string using the (const char*) operator.

@daniel

If I recall correctly something funky was happening with the connection object when I first worked on this stuff. Basically the parameters to one of the connection callbacks wasn't getting set to a default value, which didn't work very well using these wrapper types. I can't remember which one it was off the top of my head though.

#33 -- duplicate --

#34 @James,
Someone mentioned in another thread about the shaders being called every frame. Is this something I can help make more efficient using memoization of function calls? This would be a general solution for TS that I would like to help fix. I have had some success using memoization of console function calls from ScriptT3D. I am actually removed one step in the lookup and create a C++ object that remembers part of the lookup. So I was wondering if I can do the same thing purely in TS to make shaders more efficient. I figured I could help solve some of the performance issues.

#35 @demolishun

The script engine itself is ripe for improvements, but part of the problem is you'd really need to alter a ton of the bytecode and refactor the way local objects are referenced to get any real improvements in execution speed. Also the object field assignment could be much better instead of converting absolutely everything to/from a string (e.g. for math types), which I attempted to resolve in the arrays branch with limited success.

If you wanted to go the whole mile you'd also have to add some sort of JIT, but by that stage you're pretty much going to have to dedicate 99% of your time to maintaining the code since I don't think there are many people out there who are experts in optimizing a scripting engine.

#36 @James,

It keeps coming back to that doesn't it? Every time I look at this I think: Why not leverage the expert script guru's efforts by using something like V8 instead of reinventing the wheel?

I was looking at the interface of the engine to the scripting engine and thought I might be able to do a wholesale replacement if I create a Con replacement. Then I got to thinking it might be even better if I did a template to make it easier to do. I think with your work with variables instead of strings that might make it simpler. Or am I just asking for a lot of pain? Or is there a better way to do this?

Edit:
One thing I thought about was to make multiple Con implementations able to reside in the engine at the same time. I would go a step further and make it so functions and callbacks would be able to be prioritized when called. This would be an issue as it would require a lookup (that could be memoized), but it could give much faster access to every language used.

Edit:
Gah! I am getting distracted to think about changing out the language! I need to run away...run away... The rabbit is coming! Run away...run away... I must complete my 90 day proto... I must complete my 90 day proto... I must complete my 90 day proto... I must complete my 90 day proto...

#37 @demolishun

While V8 is nice on paper its lack of support for mobile platforms is a big downside if you ever wanted to port your game in the future. Now if you could more easily swap in things that would be nice, but then you introduce other problems such as:

1) Which language should be used by newcomers to make games.
2) What happens when you want to use the editors.
3) How are things mapped to each scripting language.

IMO this is a hard problem and likely needs a team of people or a really dedicated person to resolve properly. If you are busy developing your own title on a shoestring budget it is unlikely you are going to have enough resources to be able to tackle this in a good enough way to be useable by the majority of others.

#38 That is definitely true. I am sure the development of T3D will evolve over time. The bigger the crowd the more stuff like this can be addressed. This is one of the reasons I was looking into compiling TS to other languages like JS, Python, etc. Then swapping languages could be a more viable reality.

#39 @James and All,
What needs to be confirmed with the TS optimizations to be included in T3D 4.0? I want to be proactive and help make this happen.

#40 I just tried modifying some gui files and creating a custom gui profile for my project. I noticed the profile and gui changes are not loaded and it looks unstable.

Tested this on version 3 master and it is working fine with just a few issues on some files.

Submitted this issue here:
github.com/GarageGames/Torque3D/issues/479