TSE status
by Dan Partelly · in Torque Game Engine Advanced · 11/25/2005 (7:38 am) · 73 replies
Ok, is there anywhere a resource saying what works and what is broken in TSE compared to TGE features ?
Also, whats the current state of TSE development ?
Judgeing from TSE IGC 2004 screenshots (St Helen's terrain, ice cave demo), whats in CVS today , is more or less what
was a year ago. Does this means that developemnt is extremly slow or even halted, and untill TSE 1.0 is completed one
will have to wait anther couple of years ? (Considering efficient dynamic lighting and shadow algorithms implementation
are way more complex then what was planed for milestone 3 .... which seems to take over a year .... )
Dan
Also, whats the current state of TSE development ?
Judgeing from TSE IGC 2004 screenshots (St Helen's terrain, ice cave demo), whats in CVS today , is more or less what
was a year ago. Does this means that developemnt is extremly slow or even halted, and untill TSE 1.0 is completed one
will have to wait anther couple of years ? (Considering efficient dynamic lighting and shadow algorithms implementation
are way more complex then what was planed for milestone 3 .... which seems to take over a year .... )
Dan
About the author
#42
The usual thing to do, is to expect to work on an engine to get it into the exact state you want. Or live with how it is and work around any issues.
Mark: no problem with you disagreeing with me :)
11/27/2005 (1:35 pm)
I think the point is, once you understand how it all gets developed. You appreciate the way it works. Its not like a final product. But then, Ive worked with many engines which were in much worse state.The usual thing to do, is to expect to work on an engine to get it into the exact state you want. Or live with how it is and work around any issues.
Mark: no problem with you disagreeing with me :)
#43
Im disagreeing with you on many points, but discussing this is futile.
>> The usual thing to do, is to expect to work on an engine to get it into the exact state you want.
I agree as long this is fine tunning. But not a whole chapter like a unified lighting system, whose implementation
will have ramifications and require changes on many parts of TSE , including scene management, light management, shaders ,
maybe GFX additions, and depending by solution choosed acyualy impact the tools as well in some cases.
Since im a newcomer to Torque I dont know its internal structure good enough , and its working of different subystems to implement
lighting myself . I can look at a couple of months before understanding Torque good enough. And then again, lighting depends by much
more than throwing some new shaders in the engine :P
I always had a problem with ppl (not the case here so far), which where stupid enough to think that shaders support in the engine is
a panaceum. A shader dirven engine, even more than a fixed function engine , requires a shader centric aproach in many susbystems.
For example, once you go down the on full dynamic lighting model, youll have to have scene support for detrminging the set of light / renderable object interactions, cache shadow volumes (if you use stencils), deal with multiple lights lighting the same object (at least 1 pass per light here, if you use stencil shadows, if not with PS3 you could do more lights / pass), and deal with multiple light types.
Then you maybe have a need to implement shadow maps, and you doit for non-tranformable directional or spot light sources and it looks pretty good. But soon you want them to work with point light sources and transformable lights, and soon youll need to render the shadow map to a cube-map / light . Now youll suddenly fill the need to cache the shadow maps , and write a cache for them. Updating them
at every frame will trash your VRAM, you want them cached if the liugt source did not actually moved ... Complexity gorws , very fast.
(HL2 lighting model is actually simplier, but it seems that they got it right for the most part, exepting shadows. It looks sweet despite beeing a hack. HAhaha. And to think Valve have the nerve to advertise realtime radiosity as a feature of Source engine. What nerve :P)
your suddenly looking at 1 year of development if you dont know the underlieing engine base. HAhahaha. Ben and company worked with TSE
anmd they know it better than ayone most likely . So for them is way easier to fix the stuff than for me or them.
This beeing said,
anyone know current TSE incarantion in what state has the light management routines , and what light types it has support for ?
Dan
11/27/2005 (2:14 pm)
Phil,Im disagreeing with you on many points, but discussing this is futile.
>> The usual thing to do, is to expect to work on an engine to get it into the exact state you want.
I agree as long this is fine tunning. But not a whole chapter like a unified lighting system, whose implementation
will have ramifications and require changes on many parts of TSE , including scene management, light management, shaders ,
maybe GFX additions, and depending by solution choosed acyualy impact the tools as well in some cases.
Since im a newcomer to Torque I dont know its internal structure good enough , and its working of different subystems to implement
lighting myself . I can look at a couple of months before understanding Torque good enough. And then again, lighting depends by much
more than throwing some new shaders in the engine :P
I always had a problem with ppl (not the case here so far), which where stupid enough to think that shaders support in the engine is
a panaceum. A shader dirven engine, even more than a fixed function engine , requires a shader centric aproach in many susbystems.
For example, once you go down the on full dynamic lighting model, youll have to have scene support for detrminging the set of light / renderable object interactions, cache shadow volumes (if you use stencils), deal with multiple lights lighting the same object (at least 1 pass per light here, if you use stencil shadows, if not with PS3 you could do more lights / pass), and deal with multiple light types.
Then you maybe have a need to implement shadow maps, and you doit for non-tranformable directional or spot light sources and it looks pretty good. But soon you want them to work with point light sources and transformable lights, and soon youll need to render the shadow map to a cube-map / light . Now youll suddenly fill the need to cache the shadow maps , and write a cache for them. Updating them
at every frame will trash your VRAM, you want them cached if the liugt source did not actually moved ... Complexity gorws , very fast.
(HL2 lighting model is actually simplier, but it seems that they got it right for the most part, exepting shadows. It looks sweet despite beeing a hack. HAhaha. And to think Valve have the nerve to advertise realtime radiosity as a feature of Source engine. What nerve :P)
your suddenly looking at 1 year of development if you dont know the underlieing engine base. HAhahaha. Ben and company worked with TSE
anmd they know it better than ayone most likely . So for them is way easier to fix the stuff than for me or them.
This beeing said,
anyone know current TSE incarantion in what state has the light management routines , and what light types it has support for ?
Dan
#44
Since you mentioned you're new to the engine architecture, I encourage you to concentrate your learning efforts on the following:
material.cpp: Material::setShaderConstants() - You can fill in additional lighting information for all shaders here. There are already four sets of register constants set up for use in shdrConsts.h, as Ray described. It seems to me the work here is just a copy & paste job.
lightManager.h:
class LightInfo - I believe this may answer your question as to the light types supported: Point, Spot, Vector, Ambient.
class LightManager - This is your friend when it comes to figuring out lighting information. It handles all the lights in the scene, and lets you pick all of them or helps you to figure out the most influential ones. This will allow you to query all the lights you want for shaders. Your challenge will be deciding how to fit the number you want into the rendering of...
SceneObjects (ex: tsStatic, Interior*, Terrain*) - From what I can tell, TSE's lighting is strongly in favor of passing the light information off to shaders. If you feel your objects lack any sort of nice look to them, you might try mapping a material to them--you may find that it was rendering full bright because it lacked the rendering instructions to do otherwise. Run a search for SceneGraphData and you'll see where each type of object takes the lighting information from the client scene graph's light manager and passes it on to the material system, which in turn passes it to the shaders when the object is rendered.
Essentially, each object type (shape, terrain, interior, etc.) handles its own shadowing. The good thing about this is it allows you to pick the methods each will render with, which from your posts seems to be your intention. For instance, Shape-derived objects have historically used shadow volumes (shadow.h/cpp) and rendered them on top of the terrain and interiors (see Shadow::smShadowMask). Some have preferred stencil shadows for their Shapes. I haven't done that yet, so I can't help you with the specifics there, but I'm sure someone around here could if you ask nicely.
Basically there's a good deal of infrastructure in place, albeit much still from TGE, but the data is the same. What changes is how you handle it for the specific case in your game. I'm sure we'll start seeing more lighting examples coming out sooner rather than later.
11/27/2005 (4:38 pm)
Dan,Since you mentioned you're new to the engine architecture, I encourage you to concentrate your learning efforts on the following:
material.cpp: Material::setShaderConstants() - You can fill in additional lighting information for all shaders here. There are already four sets of register constants set up for use in shdrConsts.h, as Ray described. It seems to me the work here is just a copy & paste job.
lightManager.h:
class LightInfo - I believe this may answer your question as to the light types supported: Point, Spot, Vector, Ambient.
class LightManager - This is your friend when it comes to figuring out lighting information. It handles all the lights in the scene, and lets you pick all of them or helps you to figure out the most influential ones. This will allow you to query all the lights you want for shaders. Your challenge will be deciding how to fit the number you want into the rendering of...
SceneObjects (ex: tsStatic, Interior*, Terrain*) - From what I can tell, TSE's lighting is strongly in favor of passing the light information off to shaders. If you feel your objects lack any sort of nice look to them, you might try mapping a material to them--you may find that it was rendering full bright because it lacked the rendering instructions to do otherwise. Run a search for SceneGraphData and you'll see where each type of object takes the lighting information from the client scene graph's light manager and passes it on to the material system, which in turn passes it to the shaders when the object is rendered.
Essentially, each object type (shape, terrain, interior, etc.) handles its own shadowing. The good thing about this is it allows you to pick the methods each will render with, which from your posts seems to be your intention. For instance, Shape-derived objects have historically used shadow volumes (shadow.h/cpp) and rendered them on top of the terrain and interiors (see Shadow::smShadowMask). Some have preferred stencil shadows for their Shapes. I haven't done that yet, so I can't help you with the specifics there, but I'm sure someone around here could if you ask nicely.
Basically there's a good deal of infrastructure in place, albeit much still from TGE, but the data is the same. What changes is how you handle it for the specific case in your game. I'm sure we'll start seeing more lighting examples coming out sooner rather than later.
#45
You understand that all TSE lighting right now is a tremendous hack? There's the old LightManager from TGE, and we pass the first light in as a shader constant. :)
We intend to add full support for a full lighting model pretty soon (the next milestone after this one if you consult www.garagegames.com/mg/projects/tse/). We haven't yet planned out firmly what this will entail although we've done a significant amount of R&D on the problem. It will probably be a PSM based solution for shadowing. It will probably support what TGE does, vis a vis, spot, point, directional. Is this blunt enough for you? :)
I realize that you have concerns about where TSE is going, but there's only so much we can say. I could make a bunch of promises but I'd rather wait till we have something largely done, rather than commit us to random things. ;)
I think I've made GG's position on this sufficiently clear. I'm done with this thread. Seeya on the forums!
11/27/2005 (8:19 pm)
Dan,You understand that all TSE lighting right now is a tremendous hack? There's the old LightManager from TGE, and we pass the first light in as a shader constant. :)
We intend to add full support for a full lighting model pretty soon (the next milestone after this one if you consult www.garagegames.com/mg/projects/tse/). We haven't yet planned out firmly what this will entail although we've done a significant amount of R&D on the problem. It will probably be a PSM based solution for shadowing. It will probably support what TGE does, vis a vis, spot, point, directional. Is this blunt enough for you? :)
I realize that you have concerns about where TSE is going, but there's only so much we can say. I could make a bunch of promises but I'd rather wait till we have something largely done, rather than commit us to random things. ;)
I think I've made GG's position on this sufficiently clear. I'm done with this thread. Seeya on the forums!
#46
11/27/2005 (9:32 pm)
One key point that hasn't been mentioned here yet is that the EA version of TSE costs $45 less than the publicized price for the final version. I think that "early bird" discount further negates any perceived obligation for GG to provide timelines.
#47
but still Ill play a bit with it. Iff nothing, ill gain knowledge about engine architecture.
James, thanks for the fast outline you did . Its usefull to me to get me booted.
Louis:
Surely you jest. For a moment, you got me here. 45$ less. How could I not see this key point ??
The only thing which governs legally relationsip of licensee and licensor is the EULA. They are not under legal obligation because the EULA doe not explicitly mention this. No 45$ not disccounts, no nothing. The disccount of 45$ is because you get then engine in a unfinsihed state, and have to bare with the devel team for a unspecified period of time to get the thing to 1.0. And because many ppl from the indie category,
which are the target market of GG , are poor enough to think 3 times before buying a new engine, and have to save with months to get a
shinny new X18000 Radeon, unfortunately. Those ppl need a "push" to invest another 150$.
An EA version which is adopeted by many ppl is very good for GG. Event at the price with less 45$. It demonstrate viability of the process,
and the wilingness of the market to adopt the new product. It brings a small revenue to GG, but the most important thing is that it demonstrates to potential investors that things are rolling and exciting times are ahead. Thats what an EA is good for in the first place.
Now, lemme go and meditate on what a 45$ disscount can gimme. I need a new diswahser machine, I think ill ask for a 45$ disccount.
Im done with this thread as well. I thank to all of you which had a part in it.
11/28/2005 (2:32 am)
Ben thanks a lot for your comments. And im glad you already did R&D on the problem. Yes, I do understand that lighting is a hack now,but still Ill play a bit with it. Iff nothing, ill gain knowledge about engine architecture.
James, thanks for the fast outline you did . Its usefull to me to get me booted.
Louis:
Surely you jest. For a moment, you got me here. 45$ less. How could I not see this key point ??
The only thing which governs legally relationsip of licensee and licensor is the EULA. They are not under legal obligation because the EULA doe not explicitly mention this. No 45$ not disccounts, no nothing. The disccount of 45$ is because you get then engine in a unfinsihed state, and have to bare with the devel team for a unspecified period of time to get the thing to 1.0. And because many ppl from the indie category,
which are the target market of GG , are poor enough to think 3 times before buying a new engine, and have to save with months to get a
shinny new X18000 Radeon, unfortunately. Those ppl need a "push" to invest another 150$.
An EA version which is adopeted by many ppl is very good for GG. Event at the price with less 45$. It demonstrate viability of the process,
and the wilingness of the market to adopt the new product. It brings a small revenue to GG, but the most important thing is that it demonstrates to potential investors that things are rolling and exciting times are ahead. Thats what an EA is good for in the first place.
Now, lemme go and meditate on what a 45$ disscount can gimme. I need a new diswahser machine, I think ill ask for a 45$ disccount.
Im done with this thread as well. I thank to all of you which had a part in it.
#48
11/28/2005 (3:15 am)
Potential investors? You have an odd view of our company...
#49
11/28/2005 (9:34 am)
Hehe.. well *I* would potentially invest.. 10 quid enough? :) 
#50
Kudos to Dan and Phil and Ben and everyone keeping it civil.
Dan - welcome to the community. You sound like you bring a lot of skills I hope you find the Garage Games product and community as productive as I do.
11/28/2005 (9:51 am)
I'm still impressed this conversation did not turn into pure flamez0r.Kudos to Dan and Phil and Ben and everyone keeping it civil.
Dan - welcome to the community. You sound like you bring a lot of skills I hope you find the Garage Games product and community as productive as I do.
#51
You wanna get nuts?! Come on! Let's get nuts!!
11/28/2005 (10:03 am)
Quote:I'm still impressed this conversation did not turn into pure flamez0r.
You wanna get nuts?! Come on! Let's get nuts!!
#52
11/28/2005 (10:28 am)
Don't forget Moore's law, Longhorn and SM4.0. Longhorn unloads stuff so TSE will have more pow'a. 
#53
Must.... resist... temptation!
Had this conversation occured on a forum outside of the GG community, I would have put money on it turning into a flamefest.
11/28/2005 (10:57 am)
Quote:I'm still impressed this conversation did not turn into pure flamez0r.
Must.... resist... temptation!
Had this conversation occured on a forum outside of the GG community, I would have put money on it turning into a flamefest.
#54
11/28/2005 (11:00 am)
For the record, I was too busy on Marble Blast to turn this into a flame fest. 
#55
Is buying an engine that is tried and true a good idea? It is if you want technology that is not new. Part of the creative process is taking something that has not been tested and doing something amazing with it. It's like buying a new (2006)used car, because it has been driven for 40k miles and has had no problems, now you can use it. NO WAY! I want a brand new car that has 0 miles and not been tested, I could have much more fun with that.
Don't get the wrong idea, you can likely create something amazing with TSE, I am just dissapointed I wont have access to bleeding edge technology through GG. To give you a better idea of why I am a little upset, the nebula device has been out for over a year, and that is pretty much the functionality that TSE is suppose to have when complete (to my knowledge). TND is free...
Keep in mind that my post is mostly here to provoke one of the GG staff to prove me wrong and make me happy. And I would also like to say that in terms of value and ease of use, Torque products will always be on top (even compared to TND, which is free, and has no ease of use).
12/25/2005 (4:39 pm)
I personally think that GG did a horrible job with TSE. They are releasing it way too late, and the technology will be old school when it finaly reaches 1.0. With that said I do still have high hopes. GG has always had things that pop-up out of nowhere.Is buying an engine that is tried and true a good idea? It is if you want technology that is not new. Part of the creative process is taking something that has not been tested and doing something amazing with it. It's like buying a new (2006)used car, because it has been driven for 40k miles and has had no problems, now you can use it. NO WAY! I want a brand new car that has 0 miles and not been tested, I could have much more fun with that.
Don't get the wrong idea, you can likely create something amazing with TSE, I am just dissapointed I wont have access to bleeding edge technology through GG. To give you a better idea of why I am a little upset, the nebula device has been out for over a year, and that is pretty much the functionality that TSE is suppose to have when complete (to my knowledge). TND is free...
Keep in mind that my post is mostly here to provoke one of the GG staff to prove me wrong and make me happy. And I would also like to say that in terms of value and ease of use, Torque products will always be on top (even compared to TND, which is free, and has no ease of use).
#56
12/25/2005 (5:27 pm)
Hint: trolling dead threads is more effective when the GG staff is not on vacation.
#57
:) Happy Holidays!
12/26/2005 (1:12 pm)
Sorry, I posted because it was a thread linked too in a recent post and I figured it would fit the old thread better than the new.:) Happy Holidays!
#58
Totally wrong IMO*... TSE is an excellent product that will have full SM 1-3 and in the future SM 4.0 when longhorn and DX10 come out.
That is totally cutting edge... it will not be outdated. If you think that then you know NOTHING about how shader technology works. TSE when it's done and it's future will have almost unlimited potential for just about any kind of graphic effect one could imagine.
I must say, that sounds unprofessional. One should never want to "provoke" GG? To what end? It's ok to complain or bring out an issue but the dialog should be one of wanting a solution and helping by suggesting
alternatives and fixes etc.
12/27/2005 (12:11 pm)
Quote:I personally think that GG did a horrible job with TSE. They are releasing it way too late, and the technology will be old school when it finaly reaches 1.0. With that said I do still have high hopes. GG has always had things that pop-up out of nowhere.
Totally wrong IMO*... TSE is an excellent product that will have full SM 1-3 and in the future SM 4.0 when longhorn and DX10 come out.
That is totally cutting edge... it will not be outdated. If you think that then you know NOTHING about how shader technology works. TSE when it's done and it's future will have almost unlimited potential for just about any kind of graphic effect one could imagine.
Quote:
Keep in mind that my post is mostly here to provoke one of the GG staff to prove me wrong and make me happy. And I would also like to say that in terms of value and ease of use, Torque products will always be on top (even compared to TND, which is free, and has no ease of use).
I must say, that sounds unprofessional. One should never want to "provoke" GG? To what end? It's ok to complain or bring out an issue but the dialog should be one of wanting a solution and helping by suggesting
alternatives and fixes etc.
#59
-Full dynamic lighting solution. This is comming, though, and I hope it will be at least as good as the lighting solution in Splinter Cell: Pandora Tomorrow.
-Physics. TGE/TSE is majorly lacking in the phsics department, when compared to what's now standard. That being said, I don't have a problem with that as it doesn't hinder me making games.
-Accurate collisions: Torque only uses convex collision meshes for object collisions. Poly soup collision seem to be standard now days, but, again, it doesn't really affect me making the games I want to make.
Note: This isn't a complaint, but rather what I think is missing from Torque when compared to other engines. Everything else is rock solid.
Also keep in mind that Torque has alot of things that surpass many other engines out there in many ways(networking, for example).
12/27/2005 (12:55 pm)
IMO, there's only 3 major things lacking in TSE:-Full dynamic lighting solution. This is comming, though, and I hope it will be at least as good as the lighting solution in Splinter Cell: Pandora Tomorrow.
-Physics. TGE/TSE is majorly lacking in the phsics department, when compared to what's now standard. That being said, I don't have a problem with that as it doesn't hinder me making games.
-Accurate collisions: Torque only uses convex collision meshes for object collisions. Poly soup collision seem to be standard now days, but, again, it doesn't really affect me making the games I want to make.
Note: This isn't a complaint, but rather what I think is missing from Torque when compared to other engines. Everything else is rock solid.
Also keep in mind that Torque has alot of things that surpass many other engines out there in many ways(networking, for example).
#60
I am meaning no offense to anyone, and what I wrote is not directed at anyone either (I totally respect your opinion Randy).
I wrote this thread after looking into the nebula code base and then looking at my EA release of TSE, and was a little dissapointed, so I sought to provoke the GG staff to give us some more information about TSE to put my mind to rest. I have looked at Ben's .plan and others, and I don't see much regarding TSE other than the occasional thread about the ATLAS terrain manager.
Oh, well. I am sorry if I offended anyone. I am truly the opposite of a anti-torquer :)
- Anthony
12/28/2005 (12:07 am)
To provoke somebody smart is to bring out their true genius. Einstien said this many times. Somebody with intelligence will take somone provoking them as an offer to make themselves better. Somone who is not will take it as an attack.I am meaning no offense to anyone, and what I wrote is not directed at anyone either (I totally respect your opinion Randy).
I wrote this thread after looking into the nebula code base and then looking at my EA release of TSE, and was a little dissapointed, so I sought to provoke the GG staff to give us some more information about TSE to put my mind to rest. I have looked at Ben's .plan and others, and I don't see much regarding TSE other than the occasional thread about the ATLAS terrain manager.
Oh, well. I am sorry if I offended anyone. I am truly the opposite of a anti-torquer :)
- Anthony
Torque Owner Dan Partelly
Thanks at least for your words. Yeah , converstation remained on the edge of the civil side =). Which is not bad, shows that ppl
can restrain themslves. Frankly I had expected it to dgenerate rapdily , but it did not.
Im not contensting that delivering MBU for TSE will not have value for TSE. Indeed it will have. Robustness is important.
In fact the only feature I miss in TSE is a unified lighting solution which would work uniform and do its job (i.e not look dull
and plat, whithout life ... ) for terrain / interiors / static meshes / charachters. When TSE will have that it will be complete from
my point of view. The rest will be history, as they say :P
Dan