TGEA future

#1

Quote:Changing Assets on the Fy
Real-time replacement of models or dif structures after changing them sounds good to me. I dont want to restart the engine for re-loading a mesh

You can do this by creating a function that execs your assets then call that function in the console or bind a key to that function. Or just type out the exec and path in the console. A reload/refresh button would be nice in the world editor though.

Showtool Pro was made in an older version of TGE so it doesn't fully support TGEA.

#2

Quote:Showtool Pro was made in an older version of TGE so it doesn't fully support TGEA.

Thanks mb for your reply. I know that Showtool has been made with TGE. I just mentioned a few ideas.

A new Showtool could support TGE and TGEA models. But I made some other suggestions too: a material editor and a dts converter in Showtool.
If you want to get a good visual quality with your material then you also need a visual tool to tweak it.

And if you create DTS files then you know that there are many options to make lods, collision, nodes, animation or flags for translucency.
A good tool to edit these options would be cool.
Not all developers have a few thousands of dollars to buy Max and Maya for exporting DTS. And other tools like the one from Blender, Lightwave or Milkshape do not support the same amount of DTS features.

A new Showtool with DTS editing capabilites and import of collada and fbx could solve this. GG only needs to support one single import tool, one single documentation.

It is just an idea to make it better. I am fine with my Lightwave exporter although the creator of it (David) vanished from the forums.
But on the other hand: What happens if this plugin does not work anymore with a future version of Lightwave or TGEA?

#3 No reason for anyone to be harsh. We all want to see TGEA grow, and our community's needs are a huge part of that.

Quote:Changing Assets on the Fy
Real-time replacement of models or dif structures after changing them sounds good to me. I dont want to restart the engine for re-loading a mesh

For DTS models, this shouldn't be too difficult, as mb notes. It is a bit harder for DIF's because of the lighting calculations that are made. However, with the shift to polysoup in TGEA and that direction, hopefully we can see some new lighting models come into fashion that can allow more fine-grained control over area or zoned lighting.

Quote:Shadows / Unified Lighting
Casting and receiving dynamic and static shadows for every mesh would be a fantastic feature. There are so many articles about shadowmapping available. Some of them use depth maps and a post-processing shader so they dont care about the number of polygons, only shadow texture size matters.

It would be a great feature, but I also want fallbacks for fixed function which were recently rolled into TGEA. While I really like TGE, I really like the code refactoring with TGEA 1.7, and would like to see some of the benefits of both engines at least on an ability list; even if not duly implemented in all projects.

Quote:ShowTool Pro
Shader and shadow-preview is a real important feature. I would love to see a little material editor there to tweak my material until it looks good. It is really no option to start the engine again and again just to look how the material looks after changing a value.

I definitely agree, though it would have to be rewritten from the ground-up with this in mind as Dave Wyand built it from the TGE 1.3 (I believe) source with TGE in mind.

Quote:Besides that ShowTool should be able to read Collada/FBX and convert to dts. All the options of a Max, Blender or Lightwave plugin could be integrated in Showtool. So GG needs only one single supported exporter (Showtool) where we can setup LOD, collision, create Nodes for mountings and similar.

I think that a Collada/FBX workflow would be a plus. Not DTS conversion, necessarily, since the formats can still lead to severe issues between modeling applications regardless of the intent of the formats; and naming and the like will still be problematic. I like the idea of data nodes that are mounted to the model and can be altered in code rather than mount nodes and the like baked into the model. I'm not sure about the design repercussions there, though. I just had that thought while reading this. That way you could attach various mount points to your assets depending on different conditions.

Quote:Constructor
I want to see and edit materials for my structures there. So it would be great to have a TGEA mode where we can see shaders and tweak their materials.

It would require a major code refactor as it is built on TGE. Plus, I'm not sure how long it will be around with the move to polysoup. I kind of hate to see it go, though. Regardless of the desire for polysoup in the indie world, it is still going strong at Valve and Epic and id. It might be hasty to toss it completely no matter how many artists complain about it.

Quote:Scripting
What do you think about a visual data-block designer? ;)
I would love it.

Very cool idea! I'm not sure if I would use it as much as I like the idea of using it since I try not to dynamically change datablocks on the fly. But I like the idea!

#4 RefreshResManager(); works to reload assets into the World Editor!

#5 Thank you David Blake for this detailed reply.
Thanks to Dave Young as well.

By the way: I did not intend to sound harsh. I am not a native English speaker.

#6 You didn't come across harsh. I was just saying that no one else needed to make harsh comments since you said to not be too harsh in your original post! Making TGEA better is in everyone's best interest!

@Dave
Man, I just learned something new! I love learning new things!

#7 I should note that I was a bit hasty in reading... refreshResManager won't clear out existing resources, so loaded meshes would stay the same as noted in the original feature request. It finds NEW files for adding to the creator tree. Sorry about the confusion.

#8 Still, it is better than nothing (and still something that I didn't realize before!).

#9 Hey Frank,
We definitely welcome any feedback and suggestions on ways to improve Toruqe so keep them coming =)

Pretty much everything you're asking has been on our list for a *long* time so we can appreciate exactly where you are coming from.

Unfortunately, the reality is that it just plain takes time to develop new features and these things have always been lower priority (than say getting shaders into Torque or adding SplashData and Lightning to TGEA (feature parity with TGE) or fixing major crash bugs on the newest version of Windows). As we knock out the higher priority things the stuff you requested has been slowly climbing the list.

That said I would like to hit on some of your ideas and give development estimates on how long it would take for GG to finish the projects. Keep in mind that I am an overly optimistic coder (see the Constructor release schedule fiasco for reference =P) and common wisdom states that you should double a programmer's estimate to get the real estimate (at least that is what my managers tell me ;).


Changing Assets on the Fly

Very cool idea and I can definitely see the benefits. It *does* suck to have to restart the engine all of the time. Unfortunately, the way the Torque ResourceManager it written, this isn't quite as simple as it sounds. It was written for a game, not an editor. This means that it prescans and precaches a lot of the data both to speed up load times and to safely "sandbox" the game (keep malicious scripts and mods from accessing data outside of your game directory).

Here and there we've bolted on some more editor specific functionality (like accessing assets outside of the game directory) but these are just patches. To properly fix this would require an extensive rewrite of the ResourceManager and hitting on all of its interactions with the rest of the game code (most of the game code doesn't have checks to see if its data is still there and valid...it assumes it will be because the loading process handles that...so it blows up in-between the deletion of the old data and the loading of the new data...you would have to hit *thousands* of spots in the engine to fix this).

Project extimate: 4 months

Shadows / Unified Lighting

Ironically we had a GG Tech Talk about our current research on shadows map at lunch today (GG Tech Talks are a way for the different parts of the company to stay in sync with all of the cool things we are each doing). Unified lighting is *hard* to do and can be extremely expensive performance-wise. Yes, shadowmapping does do a good job but it doesn't scale well for large open scenes (like most Torque mission) since the shadow map can effectively only get so big. There are ways to combat this (parallel shadow maps) and to smooth over some of the issues that low resolution shadow maps produce but they generally involve more rendering passes and more expensive sampling techniques (Variance and Exponential Shadow Mapping for example) which all costs performance and still aren't 100% problem free. The other thing that shadow maps handle poorly is a lot of lights (basically you have to re-render the scene for each light in the scene...or more than once for some implementations).

I think the idea of a truly "unified" lighting system is great in theory but we simply aren't at the point of being able to do it. Even some of the prettiest games/engines on the market right now use a mixture of lighting models to achieve the effects they want (Unreal 3, Crytek, Source). This gives the artist a ton of control over the look and performance of the lighting. Personally, I actually think the mixture of lightmapping, simple directional shading, and shadowmaps of Half-Life 2 looks *much* nicer than the wet clay looks of the unified stencil shadows lighting in Doom 3 (which imposed some pretty strict limitations on the type and complexity of the geometry you could use).

Project estimate: 12 months

TGEA Material Editor

This is actually fairly high on our list but we haven't had time to do it ourselves nor have we laid hands on someone who was willing to do it (even for money).

What I am envisioning here is a fairly simple tool that allows you to map a material to a texture name and then toggle on and off the different features that the automatic Material system in TGEA supports (ShaderGen) plus the ability to map custom shaders. It would probably have a couple of primitves you could preview it on (sphere, cube, plane) and the ability to load a model and see the materials applied to it. Originally, I would not worry about animation, mounting, hierarchy, project management or any of the other couple dozen of handy features that ShowTool Pro utilizes (basically go with a glorified version of the Player Picker in the T3D demo). We can add those on later as time permits.

This actually isn't that hard of a project (GuiObjectView + a simple materials options dialog + a couple of informational dialogs + a system for writing out/managing the material scripts).

Project estimate: 1 month

Shader preview in ShowTool Pro

This would require nearly a full rewrite of the ShowTool Pro since it is based on a *very* old version of TGE. There isn't a ton of rendering code that would have to be replaced but there is some. I also wouldn't want to release this tool without the ability to work with the Material Editor seamlessly (ideally have it be a part of it).

Project estimate: 4 months

Collada/FBX importer

FBX is actually not very good format for this since it attempts to preserve as much of the editor specifc data about the model and its animations and materials (anyone who has had the pain of trying to bring a Max exported FBX with biped animations into Maya knows what I am talking about). Also, because this format never really got a lot of traction, the importers and exporters for it are very hit or miss.

Collada on the other hand is an excellent format for this =) It is designed to be used as a game ready format and has quickly established itself as an industry standard. It also lines up pretty nicely feature-wise with DTS (which can do a *ton* of things). Ideally we would like to be able to load Collada directly into the engine with the option to convert to DTS if you are concerned about loading speed and/or file size. We are already looking into this =)

Project estimate: 3 months

Material reloading in-game

Check out reloadMaterials() (in client\init.cs). I can't ensure that this will work 100% of the time (haven't tested it recently) but you should be able to add a button to the Mission Editor to handle this for you (or add a keyboard shortcut to Torque in general).

Project estimate: 2 days (to test/fix and add buttons and keybinds)

Constructor

Again, a ground up rewrite. There is more rendering and platform code here than in ShowTool Pro so it would take longer. Like ShowTool Pro I would definitely want the Material Editor done first.

Project estimate: 6 months

Visual Datablock Designer

I've seen a few stabs at this but nothing really concrete. The actual editor bit for this should be fairly straight-forward. I would most likely adapt something like the Inspector in the Mission Editor (which has some fairly hairy code behind it). The trickier bit is actually managing the datablocks and their files (how do you automatically write out stuff *and* let people add code and dynamic properties and functions and whatnot directly to the same file (do you?). TGB dabbles in this a little with its concept of "managed" datablocks/objects so I would start there but it is a sticky issue. Take a look at the existing Particle Editor to see what I mean.

Project estimate: 3 months

#10 So let's add that up: 33 months (not even taking into account the 2X that my managers would have you do).

These seemingly simple 8 features would take nearly 3 years of development time. Yes, a lot of this can be done in parallel but there are both limits on how many people we can put on it and how effective more people can be on each of these projects (a team of more than 3-4 people starts to get in its own way and adds additional costs in management, coordination, and communication). We also have plenty of work to be done with general engine improvements and fixes plus staying current technology-wise with the industry and you didn't hit on the *ton* of work we need to do to make our editors competitive with more modern game engines that had the advantage of learning from our mistakes.

If I were to take on these projects and was lucky enough to lay hands on the right set of contractor teams to supplement our internal developers I would roughly guess it would still take me at minimum a year and a half to get *just* these 8 projects shipped.

Honestly, there are other things that are higher priority for us to get into the engine and shipped. We have started to move on some of these projects in small ways. I can't talk about much of them or the higher priority items at the moment because they are still in research and early development and we can't ensure that they will ever see the light of day. Once they are more solid you will definitely be hearing about them!

GarageGames has to walk a delicate line between what we want for our own games, what our current customers want for their games (across a *wide* range of customers!), what will attract new customers to buy our products (gotta pay the bills), what business opportunities pop up, what our competitors are doing, what resources are available internally, what resources are available externally, and what the strengths, weaknesses, and interestes are of all of those involved. It is tricky and, yes, we've made some wrong calls along the way but all-in-all I am pretty happy with what we've accomplished!

Please keep posting ideas you have to improve Torque! Pitch in and help make some of those a reality if you can! We are listening and adding them to our list (even when we don't have the time to write up a detailed response like this one). Just keep in mind that while a feature may by #1 on your list in may be #434 on some other developers list and #12 on GG's list. We will get to them eventually but you may have to wait =)

Thanks!

#11 Thank you Matt Fairfax for this very interesting reply. I understand your position well. Besides that you made me curious about the "secret" new features. But I will promise to wait patiently :)

Regards.

#12 Is integrating PhysX anywhere on the horizon??

#13 Personally, I'd rather see engine hooks for physics components so that people could use ODE, PhysX, Tokamak, Havok, Bullet, etc.

#14 Physics Abstraction Layer is your friend for multiple-physics-engine choices.

Everyone needs to get over their overwhelming love of PhysX and explore some of the alternatives :-)

Gary (-;

#15 A gui to modify and change material settings/shaders is a quicker win, and someone from the community is probably already working on it ;) It will work pretty much exactly as Matt described, the way you'd expect in a 3d modelling environment. It wouldn't be for Constructor though, it would be for TGEA for use in changing an objects materials and settings in the World Editor the way you can in Beyond Virtual/Game Core, etc.

#16 Dave,
I hope that if someone is working on this they would be sure to share it with me when it is ready ;)

Ideally I would try to keep the Material Editor as self-contained as possible so that it could be added to any TGEA-based product (like an updated ShowTool Pro or Constructor or even to the current Mission Editor). I know sometimes that isn't always possible but keeping that mindset in mind will make it easier to port to other uses/editors.

#17 I think why alot of people are sticking with PhysX is simply because now that Nvidia owns PhysX they are back intergrating it into there 8600+ cards via a driver update. Most of there future / upcoming cards will come with full PhysX support out of the box. Having a Graphics Card that can also double as a PhysX card (both at the same time) will be a huge bonus to the physic's area in games. But its still afew months off in the distance though.

#18 Yeah a material editor will have absolutely my preference :)

#19 @Thomas - I still feel that PhysX on the GPU will stink as much as the PhysX card.

We tried it with BTD... unless your building your game around what the hardware is good at .... crap tons of cubes.... it actually makes things slower. In the real world when you mix complex ragdolls, terrain, tree capsules, and triangle meshes we've found that a dual/quad core CPU is better than the latency you get with the Agiea card.

I suspect the GPU will be the same... but i'd be glad to be proven wrong.

#20 I'd really like to see a system for Deferred Shading. It seems that with the amount of redundant calculations being done in tangent-space (basic diffuse/nDotL, which is used amazingly often), that it would be much faster to store object positions, normals, and depth information into a series of textures, performing all of your calculations there. If you wanted to get really advanced, you could even apply large-scale per-pixel relief mapping at the fraction of the cost, seeing as the relief calculations are only being called once, and that's as a post-process.