Game Development Community

dev|Pro Game Development Curriculum

Torque 2D: Editor Collaboration

by Michael Perry · 03/15/2013 (9:28 am) · 113 comments





Torque 2D Editor Collaboration



Greetings everyone! It's time to talk about the 800 lbs gorilla in the room. It's no secret that Torque 2D is an awesome engine with a slick API, lots of examples, growing feature set, and amazing community. However, one of the main drawbacks is lack of visual editors. While we have been helping out with 3rd party tool support, like TexturePacker, nothing beats a full editor suite. Before we get into it, here's a little backstory...

The Road So Far

For a while, I've been avoiding getting into a detailed, public discussion about future Torque 2D editors. The reason is because the T2D team needed to discuss a few things internally:

1. What would be our role in the effort?
2. What are potential paths we can lead folks down?
3. What initial research needs to be done to kick off the discussion?

Well, we have finally answered those three questions and have determined now is the perfect time to open it up to all users. Melv and I have had several discussions behind the scenes, so this isn't a whimsical blog. We've gone back and forth on different options, but now we need everyone's feedback.

Editor Shell

Feel free to correct us on this, but we came to the rough conclusion that it is not feasible for GarageGames to make all the editors. Nor is it likely that the entire community can organize and spearhead the effort without our help. This is based on history, our own experience, and the current size/knowledge of the existing T2D users.

What we discussed is the concept of an editor shell. The gist is that we would develop a barebones program that would support plugins. The plugins could be developed by anyone, then easily integrated by an end-user. The hope is that multiple plugins would make up an editor suite, without relying on GarageGames or any other sole entity. Our plan was to release the shell for free, under the MIT license, and allow anyone to do what they want with it. People could submit free plugins or sell them, without restriction. A plugin could be as simple as a sprite viewer or as complex as a level builder.

Unfortunately, that idea was not the tough problem to fix. The real issue is deciding on which technology should be used to create the shell. We discussed three potential options:

1. Torque GUI/Script
2. Qt
3. wxWidgets

We quickly eliminated wxWidgets. That left either Qt or the existing Torque 2D GUI/Script system. Either tech offers some major hurdles. Qt is our preferred framework for generating an editor without question. It would enable some very nice features, not least of which would be consistent cross-platform experience, window docking, plug-ins and the use of ECMA-JavaScript for extensions. We could even integrate things like Scintilla for script editing and debugging right inside the editor itself. 
 
The major hurdle we're facing right now is modifying T2D to work in a multi-viewport environment as would be required for an editor. T2D is setup to be a single-window framework. It needs to be more like T3D with a multi-window, texture->device setup, but work without problems on all existing platforms.  Without that, the only option is to generate an editor using Torque itself and TorqueScript, something that we'd do but with great, great reluctance. After all, TorqueScript was designed to control games and not larger-scale applications like editor suites. Small tools yes, scalable editors, not so much. 
 
One major failing of that approach is the GUI system which is inconsistent and just plain bad.  Additionally, the font system is pretty old.  A complete overhaul of the GUI system to include powerful layout for multi-resolution devices and the use of a font library like FreeType2 would be one possible solution to use T2D itself.

Now

Well, you have all been asking about them. Here is our reply. Now it's time to discuss! We obviously would like to hear preference, but we are really hoping we can get some back and forth going on the technical details. What would be amazing is if we could even get some folks to speak up on possible collaboration, because even the editor shell is no small effort. So what say you, community?

Regards,
Mich
#41
03/17/2013 (8:14 am)
@William, I am proposing the *opposite* of focusing given that this is open-source. If we see approximately half of the people wanting one way and half the other, nothing says everyone will ultimately agree on a path.

I'm all for discussing it, but in the end, open-source is not a democracy, but a bazaar. I'm not sure that the end of the discussion is everyone agreeing what will be enjoyable to work on.
#42
03/17/2013 (9:33 am)
@Charlie: People are going to do what they want already, and that's great. But, the steering committee does need to come up with a focus that they can try and push the community towards. Otherwise, why have a steering committee at all? Just let the community run off and splinter, and produce what they produce.
#43
03/17/2013 (11:00 am)
A Torque 2D Script editor would be awesome, because torsion doesn't work on macs.
#44
03/17/2013 (11:03 am)
@William, you and I are going a bit off topic -- the topic being which way to tackle editors -- but I *did* start it, so I'll give a quick response.

I want ALL these things and this is how I think we can get them. We could debate the way that gets people the most involved but in my experience it is the freedom to work on what interests individuals. There are thousands of potential developers who would love to contribute to a free 2D super-engine (including making it more modular). We just have to find them. I'm not too worried about that.

Of course leadership is required, but this can be on a per project basis.

As for steering committees, could it be that steering the community towards open participation and sub-projects is best? It remains to be decided. I'm willing to go either route. Just throwing out an idea (about editors) to see how others respond!
#45
03/17/2013 (5:39 pm)
Given the fact there are people on both sides (so to speak), that being Torque Script + GUI and Qt, if both paths are viable by the community, why not push both? Yes, it's taxing, and requires work.. But if enough people are willing and able to contribute, than why not have the best of both?

I think the GUI interface from TGB would be a good start-up and just overhaul the components that are in need. I'm not totally clear on the problems that are present with it currently.. but the GUI system is used throughout so many games, it would be ideal to update and upgrade it for usage, and if it's good for editors too.. than that's a bonus!

Qt, as I'm not familiar enough with it, would be set for those with better understanding of it's components and to those that understand the "how to" integration.

All in all, a long term set of tools is generally a more ideal solution, but people want things "now".. so maybe get a setup working, and than expand, meanwhile having the newer more long term viable solution in works and just keep it moving.
#46
03/17/2013 (8:35 pm)
Quote:All in all, a long term set of tools is generally a more ideal solution, but people want things "now".. so maybe get a setup working, and than expand, meanwhile having the newer more long term viable solution in works and just keep it moving.

I have given this a lot of thought over the last couple of days as it means a lot to me since T2D is my main tool for casuals. That being said, I'm inclined to agree with the above statement. Solely because overhauling the GUI will benefit both us now and the engine itself for the future. It makes sense to me that if we can achieve both good quality editors and at the same time build a better API to work with then for the sake of T2D that's the way it should go.

In the end, a git repo could be setup for the long term creation of a QT based tool chain. Judging by all the "I'm not familiar with QT" this blog has, it would seem unless some folks step up out of the wood work that deciding to go that route maybe a long long road to travel.

So my vote goes to improving T2D by rebuilding the GUI system and from it allowing a new set of editors to be born.
#47
03/17/2013 (10:00 pm)
Quote:
That said, if you got a script-level plugin system implemented, like Richard pointed out, I'm fairly sure people with the older T2D licenses could just peel out the basic editors from there and port them to the plugin interface and make up a LOT of ground rather quickly.
I'm not talking about script-level plug-ins - that's pretty much handled by the module system as-is. I'm talking about a .dll that can register a new console type that can be accessed using scripts. So you could create an extended Sprite class, have your module load the .dll via the Plug-in Manager and then create new MySprites in script and use them seamlessly as if they were part of the engine.

The Manager part is functional now, but I haven't tried to actually manage anything with it yet. The next step is to make a plug-in and try to get it to load and register properly with the Console. Once I get that working I want to then make sure that my objects un-register correctly so that when you unload a plug-in it will actually remove its namespace and sim dictionary entries, leaving the system as if the plug-in had never been there.

This sort of thing could be used in games or editors - so not only do you get to extend things via script, but you can create editor add-ins or game extensions using a native code plug-in system.
#48
03/18/2013 (2:59 am)
If you build the editors in T2D you will need to improve the GUI system. I've been digging into the old editors in the past, and it's a mess. They've implemented a weird message-passing system entirely in TorqueScript, and the code is duplicated in several places, with some portions not necessarily in logical places, and lots of magic keywords spelled out rather than referenced. If you want to replicate them in the new engine you're going to have to wade through the spaghetti on your own. And things would have to happen in sequence - the GUI systems needs to be improved FIRST, or you'll be rewriting the editors all the time.

Go with Qt (lower-case t) and you'll be able to develop in parallel with any improvements to the GUIs, not worrying about T2D breaking your editors.

On what editors are needed: Joe already said much of it, but I'll list my wishes and reasons.

The Tiled importer is looking pretty good. If it can do different perspectives and Tiled supports free-form placement of variable-sized scene objects, it's a splendid solution for now. An integrated editor like TGB is of course very desirable down the road, but Tiled and a little task-switching is a decent enough solution.

However, it doesn't support more than map placement and tagging objects. There's no particle editor (we'd definitely want that), and no paths (a nice T2D feature), no asset management (I want graphics and sound to be marked in maps so that the engine can attempt efficient loading as needed) and no integration with any sort of text display system (shitty font-based text or clever TorqueScript workarounds :).

Level editor, asset management and particle editors are the *bare minimums* we need. If you say you can manage hand-editing assets for your projects, I'm not going to work with you ;)

An asset editor would require sprite sheet handling, as clever as possible (see TexturePacker), animation, possibly bitmap fonts (see BMFont, Glyph Designer, integration with TexturePacker for examples) - and SOUND! This has been a missing component in TGB forever. 13.4 billion years of no sound asset management in TGB, and 3SS was cancelled. The asset editor should handle sound effects and music, and putting it in levels, on triggers etc. In other words, it would ideally be an integrated part of a level editor.

Since Qt is the one choice (no, really, it is - its IDE just needs to run a build on each operating system and you're done!), we're not far away from at least integrating a simple source code editor, perhaps even supporting rudimentary debugging. You know it would be nice to have everything in one editor, with multiple tearaway tabs splattered all over your 12 monitors.
#49
03/18/2013 (3:12 am)
Not wanting to side-track the conversation but quickly...

I just wanted to pick up on something there ... when you say "asset management" you mean asset-editing and manipulation right?

It's just that to say there's no asset management, it kind of sounds like you're overlooking the fact that there is, even for audio.

Any static references in a Scene are loaded immediately and for dynamically used assets (assets that are used in code and are not referenced in the Scene TAML file) you can now add them to the Scene as pre-loads so they get loaded-up when the Scene does - see recent change in dev where you can do "%scene.addAssetPreload( "blah" )" etc.

</ServiceResumes>
#50
03/18/2013 (5:48 am)
@All - Hey everyone. I read the posts over the weekend and re-read them just now. Rather than replying to each, I'm just going to dump some thoughts that address some standouts:

1. Several have stated that an overhaul to the GUI system to support editors generated by TorqueScript would be beneficial to the overall engine. This is true, however it seems many are dismissing the benefit of reworking the engine to run inside of another application. This doesn't just get us running in Qt.

As I stated before, doing so would make it much easier to port T2D to other platforms. Next to editors, the next top voted request is support for Android and/or Linux. Going the route of overhauling the GUI will benefit the game and the TorqueScript editor, but it does little to help going to other platforms. Likewise, changing the engine to run in something like Qt does little to improve the current GUI system.

I just do not want anyone dismissing either option as beneficial to the core engine. They both have pros and cons.

2. Either option is likely to break backwards compatibility for everyone's game projects. Imagine that it takes 6-12 months to complete the changes and editor shell. When that rolls out, either all your GUIs have to be rewritten or your scripts that handle rendering are going to need updating.

3. While there are some ideas we did not consider, most of your comments are opinions that provide information that Melv and I already know. They are good opinions and help us ascertain the desires of our users. That's valuable information. However, we need to get closer to a solution or this dialog will be for nought.

4. This is just repeating what Melv just posted. Torque 2D already has asset management. The interface is already there, but it's all in API form. I believe what everyone is asking for is a visual representation of the asset management, with manipulation happening in a GUI, instead of TorqueScript.

5. Oh the TGB editors are just gonna keep coming up, aren't they? Here's the deal. We started with the iTorque 2D version of the editors for 3 Step Studio. The first alpha was mostly just a customization of those editors. It was so difficult to maintain and extend, we decided to scrap it entirely and create a new interface. The more we improved the engine, the less useful the old editors became. For example, I wrote the GUI to display assets completely from scratch. There was not a shred of usable code from the TGB editor that helped complete that task, with the exception of displaying thumbnails (yay, three lines of code!).

I'm telling you now that if anyone wants to try and port the old editors, you are facing two huge hurdles. One, you're not going to get direct help from myself. I'm just not going to go through that pain again. Two, those editors and that code is still protected under the legacy T2D/iT2D licenses. If you want to port the editors, it will have to be a private project shared only with other licensees of the legacy technology.

Now, if you are telling me you want to reproduce the interface because you liked the layout and work flow, that's a different story. That is a valid opinion that can be taken into consideration. It would just have to be written from scratch, regardless of the framework chosen.
#51
03/18/2013 (5:54 am)
6. When I talk about changing the architecture to get T2D running in Qt, that does not pigeonhole the engine to just Qt for editors. That kind of change makes it possible to use any of the alternative options posted in this thread. Some extra care might have to be taken to prevent it from only embedding into Qt, but essentially the core change might make it possible to embed into editors made with wxWidgets, Python, or Adobe Flex.

7. I've seen 3 Step Studio come up a couple of times, even by my posts. For Torque 2D MIT, we have reused several art and GUI elements, but that's it. As for code, we cannot bring anything over from 3SS. It was 100% geared toward an audience that does not represent the T2D MIT crowd. While I pushed for some subeditors to stay generic so they could be reused, that battle was lost over time. More so than the TGB editors, there isn't really anything from 3SS I would reuse except for maybe the Asset Browser code, which would have to refactored anyway.

8. Finally, remember that the offering from GG was to help develop the editor shell. It was not to help build any of the plugins, like a level editor or asset browser. Trying to do everything is too much for one small team. In fact, there will likely be a call to help even with the editor shell. That's where collaboration comes in.

Now, my next post will try to get us closer to working on an actual solution.
#52
03/18/2013 (6:02 am)
Everyone is welcome to keep posting their opinion about which framework to go with, as well as what they perceive the pros/cons will be. However, I'm throwing out some more important questions that we will need answers to before proceeding.

First and foremost, who wants to contribute? Specifically, who will be willing to contribute time to help develop the editor shell? We need coders, artists, and even some UI designers. Regardless of what we go with for the editor shell framework, we're going to need some help. It's not just a matter of need, as the outcome needs to be acceptable to users outside of GarageGames. If developed in a vacuum, it may only be productive to us (GG). As I said before, we will not be developing all the plugins to make up an editor suite. That means the editor shell needs to meet the usability standards of the people who will be taking that work on.

If you are willing to step up and help out, then once again you will need to state what you would be able to work on. If you are a fan of using Qt, then will you be willing to work on a Qt project while someone else reworks the T2D architecture? If you are a fan of the TorqueScript approach and have experience creating a GUI system, would you be able to work on that while someone else works on the script implementation?

For EVERYONE, what is the acceptable timeline? I know this has already been stated, but let's get it consolidated after this post. Do you want the editor shell in 3 months, 6 months, or 1 year?
#53
03/18/2013 (6:06 am)
Oh! One list important piece that some of you have really started to understand, but I feel it's worth repeating in my own words. In no way does this work prevent anyone else from blazing their own trail. MIT license gives you the freedom to do whatever you want. You can write all your own editors and sell them independently. There are only two basic restrictions:

1. GarageGames owns the Torque brand and legacy technologies. If you create editors, they will have to be your own IP. You can create the "Jellybean Editor Suite for Torque 2D", just not "Torque 2D".

2. If your resulting work does not comply with the standards and mission for the core Torque 2D MIT engine, we would not push it into the official master branch.

OK! Continue discussion!
#54
03/18/2013 (7:14 am)
  1. I'd like to put forward support for using QT for the editor. I like the idea of making Torque 2D even more flexible and open to more platforms.
  2. I would like to contribute to it, but would like to know more about contributing and how people normally go about carving out portions of their day to accomplish awesomeness. Luckily I have some python experience if that truly is integral to the QT experience.
  3. I'm really enjoying toying around with the current engine and I can really wait for everything to be done right, especially if I have the opportunity to contribute. So I'd say lets do it right in a year
#55
03/18/2013 (8:24 am)
I have posted a link to this on the level helper forum and maybe enough people can get together and convince him to support torque mit. People are having good results with levelhelper and sprite helper. The only issue i see right now is that its Mac only at the moment.
#56
03/18/2013 (8:46 am)
I'm definitely willing to help, my general area of expertise is with coding/scripting, and I know the GUI system pretty well. I'd be willing to work on either/or (or both if needed)...
I don't mind waiting to get the next generation that will hold up for the long road, but there's a lot of people wanting things right away. I guess I'll start looking into Qt, and see what I can learn. I can't make a good enough case argument without really understanding it's capabilities, but if it's a viable solution for all needs, than it should be the route to take.

For now tho, I'll commit to helping out the GUI system using Torque script, and whether it's coding the mechanics, or getting the layout portion prepared, I'm game for it.
#57
03/18/2013 (9:10 am)
@Micheal: Just to get things moving along, I'll lend my time to the editor shell development (whatever the shell decision is).

I'll also put my vote towards a QT shell. QT is a very rich UI framework, and of course will always beat out any GUI from a game engine. You also make a great point about a QT shell forcing the engine to become more module and easier to target other systems.

I would rather make sure things are done right, and not try and please people right *now* with some hacked up TGB editor port. T2D can already be used to develop any game you want, you just have at least be able to develop with TorqueScript. If you can't do that much, I really don't see how you'd finish a fully polished game even with snazzy editors.

Hopefully someone picks up the torch for a T2D GUI overhaul though :). I also still strongly believe T2D needs a C++ plugin system so that a pluggable editor can really shine.


#58
03/18/2013 (4:47 pm)
I concur with the idea to use Qt for the editor. While it is a worthy goal to improve the GUI system in Torque, and should definitely be done, a game GUI and a UI for something like an editor suite are vastly different beasts and trying to shoehorn the ability to make a robust and easily extensible editor framework into a game GUI system is a bad idea in my opinion. You'll either have to live with a crappy, clunky and unresponsive editor, or you'll be wasting a *ton* of resources re-inventing complex controls that already exist in Qt and other UI frameworks, with little benefit for game GUIs.
#59
03/19/2013 (5:46 am)
@David Helmer,
Qt and Python are separate entities. I am just glad the Qt people have a pre-built Python library. Qt supports a lot of different languages. Sorry for the confusion.
#60
03/19/2013 (8:06 am)
@Melv

Yes, asset management in the sense that there is a way to manage the assets inside the editor. Preferably using some standard UI functionality (ie drag and drop) that automates the process of including the assets in the resources using them.

@Michael

It might have been just me but I probably missed your comment about whether an engine overhaul has a greater priority than the GUI editor.

I'm willing to contribute - I have a bit of UI design but I've not done a lot with QT. What is the intended operating procedure for this development collaboration?

Also - does using QT require the Torque2D engine to support multiviewport rendering? If enabling that feature is costly, are there other UI frameworks that would be easier to integrate?