LuaPlus binding with Tilde debugger.

#41 Wow. I'm concerned that my point may be lost in all this lua talk. Let's take Lua off the table for a moment. What I am really suggesting is that by moving all of the scripting Macros into a file or files separate from the c definition has benefits. One of which is implementing various other types of scripting.

James are you concerned that my code would make it into the main branch. I know you feel somewhat negative towards it. Why don't you and I team up and make it better?

Rich: That is a good suggestion. I'll keep it in mind.

#42 James - do you mean I display contempt in general, or in specific cases such as whenever we talk about TS? I apologise if I come across too strongly. I do have a lot of frustration with certain areas of discussion. In response to your specific question, I do have clear memories of people complaining about the scripting language, but I thought they were on the 3.6.x release posts I did on Reddit, and going back I can't find them, so I'm not sure where they were. The last thing I'll say on the topic is that my friction with people over TS is that to me, it clearly seems like a terrible language. I'm fine with people suggesting, like you are, that replacing TS is impractical. That's a reasonable and important consideration. What really confounds me is that people say they would prefer it as a language, which I cannot understand.

Would I be right to say your recommendation is to continue with TS as it currently exists and not worry about a replacement or additional language bindings? I can live with that, subject to a couple more tweaks (for example, anonymous functions would be helpful, and appear to be fairly easy to implement).

Antony - did you consider implementing the high-performance code in C++? That tends to be the usual strategy when working with Torque. It seems like it might be a more direct solution than implementing another scripting language!

#43 Dan: If performance were all i was after c++ is the way to go, but I would also like hot loading code and threading. The the hot reload for improved iteration times. I'm thinking torque might already be able to do this? i'm not in any stress over this stuff. I just wanted to contribute. I also like the separation that scripts give from engine code. Lua also has quite the history in terms of use and optimization. Crytek and so many other games. it sounds like people aren't ready for the changes i am proposing. it is a lot of work. i get it. no biggie. ill probably just use a lua 2 Torquescript bridge. its less stress for me. thanks everybody for the consideration.

#44 @Daniel

I was referring more to your "it's really not worth our time and effort to maintain this engine solely to serve the interests of the 50 or so regular users that are active here" line.

Language preferences are probably as old as computing itself, so I shall not dwell on that here.

In the end, I am just being realistic here. From what I've seen there are a group of people who tend to go "TorqueScript sucks, lets replace it with <my favourite programming language X>" but when it comes to actually doing anything very few people are actually interested in doing anything concrete about it, and nobody else seems to share the same opinion to form a consensus. There have been quite a few discussions over the past two years which have all largely devolved towards inaction on this front.

On the subject of continuing with TS, my preference would not be to continue with TS as-is, but rather to gradually improve it like any other engine component. With the exception of a few deviations, this has been my general approach in any case.

Additional language bindings are fine but as I mentioned I personally don't see much of an advantage towards doing it. Lets take Unity for example. You can write in C#, JavaScript, or Boo. But everyone I know just ends up using C#.

In any event from what I've seen the steering committee as a whole seems very reluctant to commit to any particular approach on this which I feel doesn't particularly help in the long run since it brings the existence of the current console system into question. If for example it's going to be ripped out it's likely not worth my time trying to contribute improvements. Similarly in Antonys case if his work is just going to end up being completely pushed aside, then it's arguably not worth his time trying to push it.

@Antony

Don't throw in the towel yet. This is only the beginning. I'd be quite happy to suggest improvements to your binding system, I'm just being critical here since this is a fairly core part of the engine experience :)

#45 another 3 pages of pointlessness.

the OP suggests that complete removal of TS is the *only* way forward but in the actual OP states that doing everything TS does will effect performance.

This thread of full of people who hate the way it but dont want to make any effort to fix the *whole thing*.

As for people who prefer TS? maybe its the people who use it day in day out,. watching in bemusement all those who bullshit claim and bullshit counter claim that every other scripting language is better, only nobody knows how to get that working in the engine.

nobody has yet explained how any other language will be better than TS other than a small % of speed increase.

As far as i can tell the only major advantage will be for those people who write too much in script and try idiotically to get a script language to do all the heavy lifting

If your game is shit because of torquescript, god himself could create a perfect scripting language, your game will still be shit

stop blaming the damn scripts, if the scripts are slow, write a source function, or get somebody who knows how to write a source function.

seen this discussion on and on for the past 2-3 years or more, the discussion itself is counter productive a waste of time for everyone involved. much like every other discussion on exactly the same topic over the years.

Perhaps the time should be spent making TS better; spend some time fixing the optimisation routines just so that the "lets replace TS with xyz script" discussions are even more pointless and time wasting than they are now.

#46 Personally (and yes im biased) it needs to be handled like OMNI where TS is left intact but its made where u can ADD another language to the engine. We chose C#, but it could be anything.

There is no real need to rip out TS. leave it be. ripping it out will just shred the community more.

#47 On the overarching discussion:
*grabs his hatrack*

The scientist in me says every inch of performance we can dig out is a Good Thing (TM), and we should by all means be using the best, most precise tools for the job.

The engineer punches the scientist in the gut and tells him he needs to just do what gets the job done, whether that's accepting limits, tweaking the current language, or throwing in a second for specific use cases.

The sociologist sits on the both of them and points out that replacing the old language, or adding a purely vestigial version of a new one without thorough conversion samples is a good way to turn the place into Babel.

The Team Lead grabs all three by the scruff of the neck and tells them to pick a direction and stick to it.

and finally the Opinionated Coder grumbles to himself in the corner that if you feel it's something that needs doing, keep at it, give the occasional status update, and see if more folks adopt the fork naturally. It is in fact, where most of the engines resources came from, after all.

*drops mic and hat-rack*

#48 *applauds Az*

#49 Wow, this turned out to be very unproductive. I actually had a great deal of work done. So I'm not talking out my butt. I was not planning on removing TorqueScript but providing a layer of abstraction so that other languages were optional in the future. I've learned a valuable lesson - this may not be the proper forum for a conversations about this stuff in the future.

#50 No need to get your feelings hurt over this - it's how most of these discussions here go. A bunch of old hands will always jump in because (as someone has mentioned) inevitably someone is doing something that has been done before.

Your best approach is to broach an idea, watch until the smoke clears and then glean the useful input from the schlock. Never take anything said here personally. EVER.

Every endeavor carries risk - pick your comfort level and drive on. This particular task is a large one and ultimately only you can decide how much of your time and effort you feel is "worth it."

#51 Honestly, you *do* seem to be on the right track as far as general direction goes. (and really, Lua's about as close to TS as they come in terms of syntax translation). Again, the biggest choke-point for folks is likely to be the bit about both shifting calls and the code that comprises them simultaneously.

#52 Antony:

Sorry, when I was talking about the macros before, I was THINKING of the engineAPI stuff, but was calling them macros because I'm kinda dumb sometimes, haha.

But the DefineEngineFunction functions and whatnot from the engineAPI stuff strikes me as a good way to generate those bindings semi-automatically, with the added bonus of being able to let the engine generate docs for the script-exposed stuff at the same time.

Not knocking your direction, but I am curious why you don't like that method. Could you explain?

#53 Antony - Unfortunately it's the only place we have, but given you were asking whether your work would be pulled into the trunk, we need to have a public discussion on the benefits and pitfalls. For now, your solution seems to involve some code duplication and more maintenance effort, unless I'm still misunderstanding it. For those reasons I don't think this would be officially accepted, even if we should decide to move ahead with multiple scripting languages (or even a single non-TS one).

James - ah sorry, I get you. I didn't mean to imply that there's some cabal of users who want to bend the SC to their will. I just think some topics are an inevitable conflict between current users who are already comfortable with how the engine works, and new/potential users to whom that way might be perplexing of offputting.

Also, I completely forgot about your 'engine experience' question. I meant to rebut, in general, the idea that the current 'engine experience' is something we should be striving to maintain.

How about we do this. Start a wiki page cataloguing the various options we have regarding engine scripting, and make it a comprehensive list of pros, cons, work required (and maybe even who supports which).

Also, Az has mentioned that he's not really sure what my actual complaints with TS are, despite me being sure I'd expounded on them many times in public. Maybe not, so I'll have my rant and write up a bit list of what I feel is wrong with it so I can get it off my chest :P. Whether that provides inspiration for someone who wants to fix issues with TS (if they agree with my opinions), or just makes it clearer where I'm coming from, who knows.

#54 OH MY GOODNESS! :) I really must suck at explaining things. I would like to emphasize that the duplication would only be temporary until we removed the Macros out of the definitions. And POOF no more duplication. Ughhh!! I don't mind getting shot down but it would be nice if people understood what i was talking about. C'est la vie. :)

#55 Wait, but don't the macros just get moved? So if I then wanted to add a new method, which was exposed to both TS and Lua, I'd then have to add a TS macro, and a Lua definition, myself?

#56 The idea was to move the macros into their own files separate from the definitions then point the macros to the static functions in script bindings that lua is also pointing to. voila only one set of static functions that 2 bindings are pointing to. if you want later you can have python point to those same statics. voila 3 scripting languages sharing the same code.

#57 i'm approaching the second phase which is extract the macros point them to the static functions in the scriptbinding and consolidating the code.

#58 I'm estama from BeamNG.

First of all, thanks Antony looking into the subject of LUA + T3D integration. My 3 cents into the subject follow, with a big IMHO at the beginning of every sentence.

About Torquescript (TS). TS is very badly designed from a programming language point of view. It is also very slow, from an implementation point of view. In addition not many people know TS. Nevertheless TS is here now and it works, so we should try to keep it in parallel with whatever solution we come up with, until it is eventually (and gradually) replaced.

The replacement should at least try to solve above problems. It should be well designed, have wide adoption and be extremely fast. Also due to the nature of GFX engines, it should work nicely in a multithreaded environment.

The multithreaded constraint, necessitates a Virtual Machine (VM) that is small enough so that each thread will be able to run its own VM. IMHO this precludes any of the heavy weight VMs like C#. LUA is one of the few languages that put a heavy focus towards having a small VM.

For the "extremely fast" problem, LUA offers LuaJIT which has one of the best (if not *the* best) tracing JIT engines. This is what allows us in BeamNG to have a quite big part of the physics processing being done on the LUA side. LuaJIT being relatively close to C++'s speed enables a larger percentage of the C++ code to move to the LUA side. Code that would be prohibitively slow if it run on a slower VM.

In addition to above, LUA is nicely designed and already has wide adoption.

Finally i would like to point to some other very nice design commentary about binding LUA to a GFX engine:

http://bitsquid.blogspot.gr/2013/02/why-lua.html
http://www.bitsquid.se/files/lua.html

#59 Is there any way to actually measure the speed of lua vs torquescript in a Torque3D game?

#60 LUA's VM has had a lot more optimizations by people that know about VMs [1], compared to TS which has a simplistic VM.

With LuaJIT the difference is even more pronounced. There are (micro) benchmarks where LuaJIT is very close [2] or even surpasses C's speed [3].

[1] http://www.lua.org/doc/jucs05.pdf
[2] https://attractivechaos.wordpress.com/2011/01/23/amazed-by-luajit/
[3] https://gist.github.com/catwell/6389992