1.4's DynamicTexture and GUIs

#1 Yeah, make a GuiOffscreenRenderCanvas or something, and then when that gets updated it renders to a texture instead.

#2 Looking at GuiCanvas for the first time it seems too complex to derive from, so a GuiOffscreenRenderCanvas would be just derived from GuiControl and implement the parts of GuiCanvas that we care about for an offscreen surface.

It might even make sense for GuiCanvas to keep track of all GuiOffscreenRenderCanvas objects and update them itself. This also makes sense since GuiCanvas is the one to process input events and could have the added capability to push input events to the active offscreen canvas.

Still i wonder if this is too invasive to the whole system. I guess i should look around some more and see if there is a simpler way to fit this in.

#3 Tom and Pat - Slightly off topic but how about this for an interesting use for DynamicTextures??

I had a couple of hours to spare this afternoon and decided to try to get a dynamic texture object working inside the game I am making just to see what went where and what the effect on performance would be.

After setting it all up, I was trying to decide what the object should be that receives the dynamic texture in the game and what control I should render onto it to best test it out. It then dawned on me that it would open up some interesting possibilities if a T2D game was rendered inside a TGE game - actually inside rather than overlayed in 2D.

So, with an fxRenderObject (for the in game display), a copy of the T2D spaceshooter demo set up to run on an invisible fxRenderWindow inserted into the playGUI and a DynamicTexture to link them together I have a T2D game running inside a 3D game world.

And the best bit is (with a change to the demo key mappings) you can actually run up to the in game display and start playing the T2D game!

Performance wise, framerate in the game is only down about 20% which I dont think is too bad when you consider that the T2D control is rendering at 512 by 384 in the background.

Just to round off, here is a video (made with FRAPS) of me not being very good at the spaceshooter demo inside a Torque game :)

www.vhp.co.uk/T2D_meets_TGE.zip

Encoded in mpeg (plays in windows media player) and 5.2MB

EDIT
Just thought I would mention that the slightly jerky background in the T2D window is a result of FRAPS recording at 30fps - it is as smooth as the T2D demo when playing normally.

#4 Tom, I have an offscreenrendercanvas, actually...um, actually check 1.4, I think there's an effectCanvas in there unless that got cut. But I made changes to GuiCanvas to make it deriveable at least. I dunno, there's code floating around somewhere on my hard drive.

And David, that is sweet! I'm glad the dynamic texture thing worked as well as I intended it to. :P The performance hit could be way less, however it doesn't use (at all) efficent off-screen rendering because that would require pBuffers and such and that's not supported cross platform.

#5 @Pat - Ahh... i see now. DynamicTexture is more than just a texture, but also keeps track of all DynamicTexture objects. GuiCanvas::renderFrame then calls static functions in DynamicTexture to render and update all DynamicTexture objects in the system. It doesn't deal with input, but i see how it all works now.

I do not see anything called GuiOffscreenRenderCanvas or effectCanvas in 1.4.

Also DynamicTexture will eventually need some culling mechanism as RTT is one of the most expensive state changes in DirectX.

#6 Yeah, it's really not at all a general-purpose object, it was more like something I had laying around and so I stuck it in. I'll see if I can find that canvas subclass I did.

#7 Ok so i ripped out most of the stuff in Pat's DynamicTexture as most of it was workarounds for not having true RenderToTexture support for most TGE target hardware. Still the idea of having these gui textures manage themselves thru statics and having GuiCanvas call to update them is retained.

I'm also gonna derive the GuiTexture class from GameBaseData like the GlowBuffer and in particular CubemapData for how they work. The idea being that your material datablock could look like:

datablock GuiTexture( ComputerTerminalTexture )
{
  guiControl = ComputerTerminalGui;
};

datablock Material(ComputerTerminal) 
{ 
   baseTex[0] = "$guitexture"; 
 
   guiTexture = ComputerTerminalTexture;

   baseTex[1] = "static_anim"; 
   animFlags[1] = $sequence;
   sequenceFramePerSec[1] = 0.25;
   sequenceSegmentSize[1] = 0.25;

   baseTex[2] = "vsync_line"; 
   animFlags[2] = $scroll;
   scrollDir[2] = "0.0 1.0";
   scrollSpeed[2] = 0.2;
};
addMaterialMapping( "ComputerTerminal", "material: ComputerTerminal" );

Or something like that.

@Pat - If you could find and post that it would be tremendously helpful.

#8 Pat - regarding the performance of DynamicTextures, I made a few changes and now dont see any real changes in framerate due to the dynamic texturing.

I removed the storepixels() and restorepixels() calls, removed the calls to updateScreenTextures() and updateEndOfFrameTextures() to just leave the gui rendering call.

Running at 1280x1024 in full screen OpenGL :-

Base game = 200-222 fps
Base with T2D running but not rendering = 185-195 fps
Base with T2D rendering to dynamic texture = 182-195 fps

Gives me an excuse to start some T2D work now :)

@Tom - Apologies for the thread hijack - I shall cease and desist.

#9 Found it

This *should* have everything you need. Stick the vectorfield into /dgl/
It's not the offscreen canvas, but it should show you how to subclass canvas.

#10 David, great minds think alike, lol. I was planning on making this exact modification myself. I would love to see your notes on the subject if you are planning on sharing.

#11 @David - No problem... really i think these subjects are related in that IMO both should be attainable thru a similar system in TSE. I know nothing about T2D... is it based off a GuiControl like GameTSCtrl?

@Pat - Thanks i'll take a peek at it now.

#12 @Tom - The display window for T2D is a GuiControl which contains everything else and is derived directly from TGEs GuiControl class. When I read about the DynamicTexture for rendering Gui controls I was going to just try some in game Gui effects but then thought why not try T2D instead (which can itself be used to make interactive 2D GUIs). One thing, as they say, led to another.

@Gonzo - I will tidy up what I have done (it was just chucked in a 'I wonder if this will work' moment) and either submit a resource or post it to a forum. I should get a bit of time towards the weekend.

#13 Eventually I'd want to get an off-screen render canvas working in TSE using projected rays to do input events ala Doom 3 style stuff so you could have GUI's on in-game objects.

#14 @Pat - That's exactly what i'm working on. I'm just trying to fully realize it to include how an artist adds them to their shapes/interiors. It's a world of difference when an artist can add a material with a gui attached to anything he creates.

Maybe Brian or Ben can chime in with how they see this working in respect to the material system. Is what i outlined above reasonable? how broken will it be once materials are no longer datablocks?

#15 The easiest thing might be to just add another script parameter to the Material class - see Material::initPersistFields() . You could just add a gui name in there.

Then when you shoot a ray at an object, look through the material list to see if one (or more) of the materials contains a gui object. Determine which gui you hit with the ray, and then do a findObject( guiName ) to get the actual gui object that you hit.

This should be a piece of cake to port when the Materials get changed over. They will just be client side script objects and will still have script parameters just like the datablocks.

#16 @Brian - Ok... so it's just about what i planned out. I'm making progress over here on it. I should have something working by tomorrow which i'll post here.

#17 Ok i got some initial results tonight. Take a look at the screenshot... nothing too impressive, but it's a first step.

What i did was create a new class called GuiTextureCanvas (more descriptive than offscreen canvas or dynamic texture IMO). It derives from GuiCanvas so that GuiControls don't bitch about not finding the root canvas (so far at least... there are probably wins for user input as well). I stick creation of the GuiTextureCanvas class in some client side script:

new GuiTextureCanvas( ComputerTerminalCanvas ) {
   guiControl = optionsDlg; // this is really a TypeString
};

Bad idea .. good idea? I have no clue if having multiple GuiCanvas derived classes instantiated is a bad thing... someone enlighten me. I do know that all the ConsoleMethod()s for GuiCanvas refer to the global Canvas and not the object pointer passed by console method. So anyone doing ComputerTerminalCanvas->pushDialog( foo ) would get the wrong result. It seems to me those ConsoleMethod()s should be fixed to not access the Canvas global. Ben?

How this works with materials is like so:

datablock Material(ComputerTerminal) 
{
   baseTex[0] = "$guitexturecanvas"; 
   emissive[0] = true;
   guiTextureCanvas = "ComputerTerminalCanvas";
};
addMaterialMapping( "ComputerTerminal", "material: ComputerTerminal" );

When Material::setStageData() is called it checks for $guitexturecanvas and does a Sim::findObject to find the canvas and hooks up the texture. Brain tell me if there is a better place to put this. It seems it probably belongs somewhere that is updated per-frame as I could imagine the texture handle changing after material load.

So my next steps is to fix a lot of things before I even move on to input. For some reason my backgrounds for my main menu are gone now... Ummm... Bug I guess. Also there is no culling... It updates the textures every frame even if it's not visible... This is bad and needs to be work out soon. I also need to work out having the same GuiControl attached to multiple textures which are all visible in the same frame. So lots of little things to work out tomorrow.

Suggestions are very welcome.

#18 Should be ok to create GuiTextureCanvas in a client side script. Looks like we'll need to change around the Gui's a bit to better handle this type of situation with the global references to Canvas and all.

setStageData should be OK. The texHandle won't change unless the texture is deleted. Should even work through a reset (ie. changing from windowed to fullscreen).

#19 I think the GuiTextureCanvas class can be handled pretty much like the CubemapData block. It should be part of the materials which would will all be client side in the next milestone anyway.

Now to be clear GuiTextureCanvas doesn't replace the global GuiCanvas... it's a separate offscreen canvas. The only issue i see with the global Canvas is that the console methods directly access the Canvas global and not the object passed via the ConsoleMethod macro. So someone though Canvas should be a singleton in the past... now there is still a Canvas, but there may be other special Canvas classes.

The only reason the texture would change is if we have to resize if because of a change in the GuiControl size. This could possibly happen as some point, so i need to prepare for that case. Maybe it should really belong in MatInstance... maybe in setTextureStages().

#20 Before I did anything i had to implement a better interaction system for our game as the binding stuff to different keys approach wasn't a real good solution. So now i can walk up to interactable things and the script changes the meaning of mouseFire to what is appropriate for that interaction. That took a ridiculous amount of reverse engineering of the code to figure out that Armor::onTrigger is where I could do that. Anyway...

My next issue was figuring out the material I hit. Well of course nothing in the game seems to return material info for the raycasts. So I started with the one I knew best... the interiors. It was actually damn simple to add this into Interior::castRay_r:

if(inside)
{
   info->face = surfaceIndex;
            
   // TOM: This returns the material id of the surface hit.
   MatInstance* matInst = mMaterialList->getMaterialInst( rSurface.textureIndex );
   if ( matInst && matInst->getMaterial() ) 
   {
      info->material = matInst->getMaterial()->getId();
   }

   break;
}

That seems way too damn simple to work... But it does. I also added a 7th return parameter to the containerRayCast console function to return the material id to script. I can now see what materials I hit on raycasts which hit interiors.

Now, Brian you probably know, will this break once materials go client side? Will the server scripts still have a material id that it can use? Is it a bad thing to be adding more return parameters to containerRayCast... will it break something? Ideally some of this work could move into TSE head, so Brian/Ben please let me know if something i'm doing would be bad for a later inclusion via a patch.

Now my next step is to add uv hit information into the raycast... again I'll start with interiors. This should give me everything i need to know to click on gui surfaces.