Torque 3D Embedded Browser (WebKit)
by Josh Engebretson · in Torque 3D Professional · 01/15/2010 (8:55 pm) · 305 replies
Hey guys,
Here are some highlights:
1) It uses QtWebKit which is based off Apple's WebKit, the same HTML5/Canvas based web rendering engine from Chrome, Safari, Adobe AIR
2) QtWebKit is *easily* the best designed embeddable web browser I have found in exhaustive travels... (I actually compiled Chrome from source at one point, not recommended). Qt is *extremely* well supported and owned by Nokia who recently paid $153,000,000 for it. So, it is probably going to be around for some time.
3) Flash works :)
4) It is setup as a GuiControl so interacts perfectly with mouse events, control overlap, filters...
There are currently 3 rendering modes:
1) Render to Texture: This mode renders the web view to an offscreen buffer and then uploads it to a GFXTexHandle. It uses the Torque GUI system input for mouse and keyboard. As we're rendering to a texture, compositing and rendering to 3d surfaces is possible. Flash renders as long as it is set to windowless mode in the HTML (not a big deal). Instead of using Flash for anything other than presentation/video playback, I would suggest looking into Canvas/HTML5/CSS animations/effects in this mode.
2) Render Native: This mode integrates a native child window with Gui Control, giving the best of both worlds :) There is no need to render to texture as we're using an actual clipped child HWND which uses the Torque GUI system for position/extents and focus :) Transparency does not currently work, however under Vista/Win7 this should be possible. I should be able to get masking working with minimal effort, which would allow for shaped windows and transparent areas.
3) Render Popup: This mode pops up a standalone window, which is owned by the application and consistent in scripting with other modes. This is useful for things like integrated document browsing, tutorial video watching, game doc reading, game forum reading, etc... where you don't want to crowd your game interface with browser. However, you want CLOSE scripted integration with a standard browser (no worries of IE/Mozilla/Chrome), which isn't possible with the current "launch browser" functionality.
These modes work with HTTPS adding a nice way to authenticate to servers, secure purchasing, etc.
It is still pretty early in the development and the idea with the code release will be to get feedback on usability and features... especially WRT integration with Javascript on the page, etc.
EDIT: The first version has been released and is available HERE
Regards,
- Josh Engebretson
Mythos Labs, LLC
Here are some highlights:
1) It uses QtWebKit which is based off Apple's WebKit, the same HTML5/Canvas based web rendering engine from Chrome, Safari, Adobe AIR
2) QtWebKit is *easily* the best designed embeddable web browser I have found in exhaustive travels... (I actually compiled Chrome from source at one point, not recommended). Qt is *extremely* well supported and owned by Nokia who recently paid $153,000,000 for it. So, it is probably going to be around for some time.
3) Flash works :)
4) It is setup as a GuiControl so interacts perfectly with mouse events, control overlap, filters...
There are currently 3 rendering modes:
1) Render to Texture: This mode renders the web view to an offscreen buffer and then uploads it to a GFXTexHandle. It uses the Torque GUI system input for mouse and keyboard. As we're rendering to a texture, compositing and rendering to 3d surfaces is possible. Flash renders as long as it is set to windowless mode in the HTML (not a big deal). Instead of using Flash for anything other than presentation/video playback, I would suggest looking into Canvas/HTML5/CSS animations/effects in this mode.
2) Render Native: This mode integrates a native child window with Gui Control, giving the best of both worlds :) There is no need to render to texture as we're using an actual clipped child HWND which uses the Torque GUI system for position/extents and focus :) Transparency does not currently work, however under Vista/Win7 this should be possible. I should be able to get masking working with minimal effort, which would allow for shaped windows and transparent areas.
3) Render Popup: This mode pops up a standalone window, which is owned by the application and consistent in scripting with other modes. This is useful for things like integrated document browsing, tutorial video watching, game doc reading, game forum reading, etc... where you don't want to crowd your game interface with browser. However, you want CLOSE scripted integration with a standard browser (no worries of IE/Mozilla/Chrome), which isn't possible with the current "launch browser" functionality.
These modes work with HTTPS adding a nice way to authenticate to servers, secure purchasing, etc.
It is still pretty early in the development and the idea with the code release will be to get feedback on usability and features... especially WRT integration with Javascript on the page, etc.
EDIT: The first version has been released and is available HERE
Regards,
- Josh Engebretson
Mythos Labs, LLC
About the author
http://www.TheEngine.Co
#3
So i haven't looked closely at this, but it seems like the key feature is gonna be redirecting the render-to-texture method to a material texture input.
If you've seen the PostEffect and new Material stuff Pat came up with this idea of doing a prefix of pound sign # for special textures. So we've always supported like.... $backbuffer, $cubemap, etc. He added the idea of like #gbuffer, #lightinfo, and others for special named texture targets that users can define.
You could hook into the same system which allows us to do something like this...
... it won't get you any automatic mouse input interactivity, the GuiWebCtrl would never be pushed on to the canvas, and the rendering would happen on demand when the Material requests the texture on each frame.
Look at MatTextureTarget in the codebase which currently enables this functionality if you derive from it and implement a few methods. Hit me on IM if you need help.
Anyway... this would allow someone to then via script control the GuiWebCtrl... navigating to local or remote pages, browser script calls, etc.
01/16/2010 (2:33 am)
@JoshSo i haven't looked closely at this, but it seems like the key feature is gonna be redirecting the render-to-texture method to a material texture input.
If you've seen the PostEffect and new Material stuff Pat came up with this idea of doing a prefix of pound sign # for special textures. So we've always supported like.... $backbuffer, $cubemap, etc. He added the idea of like #gbuffer, #lightinfo, and others for special named texture targets that users can define.
You could hook into the same system which allows us to do something like this...
singleton GuiWebCtrl(MyWebControl)
{
renderMode = texture;
targetName = "MyWebControlTex"; // User gives this texture target a name.
extents = "256 256";
url = "http://www.google.com";
};
singleton Material(WebMaterial)
{
diffuseMap[0] = "#MyWebControlTex"; // Reference the named target here!
};... it won't get you any automatic mouse input interactivity, the GuiWebCtrl would never be pushed on to the canvas, and the rendering would happen on demand when the Material requests the texture on each frame.
Look at MatTextureTarget in the codebase which currently enables this functionality if you derive from it and implement a few methods. Hit me on IM if you need help.
Anyway... this would allow someone to then via script control the GuiWebCtrl... navigating to local or remote pages, browser script calls, etc.
#4
01/16/2010 (7:10 am)
Amazing ! This would be a really great key feature for Torque ! 
#5
While I'm likely the only one on planet not seeing a reason for a browser in a game, there are surely others that would consider themself waiting for this nearly as much as waiting for the FK :)
01/16/2010 (9:20 am)
Very interesting stuff :)While I'm likely the only one on planet not seeing a reason for a browser in a game, there are surely others that would consider themself waiting for this nearly as much as waiting for the FK :)
#6
Let's see...uses....
login system is the first one that comes to mind.
01/16/2010 (9:45 am)
This looks awesome. Let's see...uses....
login system is the first one that comes to mind.
#7
In game advertising on 3d objects such as billboards, secure HTTPS microtransactions (or just purchasing the game), integrated bug tracking during development (or release), Flash video/animation/minigames/UI, HTML5/Canvas/CSS animations/transistions/UI, Community integration with forums and messaging (think Steam), integration with Facebook Connect for credentials and friends lists, context sensitive/hyperlink browsing of game/editor manuals, in game marketing with links to reviews/user comments, etc.
Not forcing the user to leave your game to do any of these things... or whatever else you want to do with the web with *tight* integration/interactivity.
01/16/2010 (10:18 am)
To rattle off a few ideas:In game advertising on 3d objects such as billboards, secure HTTPS microtransactions (or just purchasing the game), integrated bug tracking during development (or release), Flash video/animation/minigames/UI, HTML5/Canvas/CSS animations/transistions/UI, Community integration with forums and messaging (think Steam), integration with Facebook Connect for credentials and friends lists, context sensitive/hyperlink browsing of game/editor manuals, in game marketing with links to reviews/user comments, etc.
Not forcing the user to leave your game to do any of these things... or whatever else you want to do with the web with *tight* integration/interactivity.
#8
I'll be looking into the material stuff, definitely for an initial release... and will see about hitting you up on IM if I am having trouble (or just to say "Yay, it works!") :)
01/16/2010 (10:25 am)
@Tom: Thanks, I am going to be working on getting something together this weekend. Right now, been focusing on the GuiControl stuff and making sure input/rendering is working there (and flexible).I'll be looking into the material stuff, definitely for an initial release... and will see about hitting you up on IM if I am having trouble (or just to say "Yay, it works!") :)
#9
Look forward to hearing more.
01/16/2010 (12:06 pm)
Josh this is pretty awesome man. I would definently want to see what kinda of progress your making with this and offer any help, testing etc. This sounds like the perfect tool for a project im working on. Look forward to hearing more.
#10
The phrase that really got my attention was "render to texture." If I'm excited about GUI web browsers, I'm literally (applauding politely -ed) over the ability to render a GUI element to a 3D surface.
Any chance this code will include support for other types of GUI objects to render to texture as well? I think we can all agree this is a feature that would get a lot of use.
01/16/2010 (8:39 pm)
This is really great news. I knew that this was going on somewhere in the background, but had no idea we were about to get this code or anything even close. The more things players don't leave the client to do, the easier it is to develop a community that's tied directly to the game experience. I'm one of those people who actually really likes a lot of what Steam does.The phrase that really got my attention was "render to texture." If I'm excited about GUI web browsers, I'm literally (applauding politely -ed) over the ability to render a GUI element to a 3D surface.
Any chance this code will include support for other types of GUI objects to render to texture as well? I think we can all agree this is a feature that would get a lot of use.
#11
01/16/2010 (8:46 pm)
@Henry - I don't think there is a current plan to make Torque GuiControls render to texture. If you have HTML then you can do the same sorts of things and more.
#12
One of the best features of this is the tiny amount of code that is necessary... and shipping a product with QtWebKit binaries isn't at all crazy or hard to support (compressed something like 11 megs, so not bad at all for a full featured browser!)
The focus on the first release is good GuiControl integration for serving up the web in your Torque 3D application. I am also working on getting pages rendered on 3d surfaces (running into a slight issue with Material on this front, thanks to Tom for the IM session).
From there, better TorqueScript integration, picking on 3d sufaces (I actually, think this is best done in 2D and the novelty quickly wears thin, but it is probably worth having), performance tweaks (rendering on a separate thread, dirty rectangle), etc
I have some other (technically more important) work that needs doing with Torque, but this should definitely be a good starting spot and if anyone wants to dive in and help, that could be cool and I can try and help facilitate it.
01/16/2010 (9:12 pm)
Thanks guys, really this was another weekend experiment that kind of took on a life of its own. QtWebKit does all the heavy lifting and if you're interested Nokia's documentation is a pretty good read.One of the best features of this is the tiny amount of code that is necessary... and shipping a product with QtWebKit binaries isn't at all crazy or hard to support (compressed something like 11 megs, so not bad at all for a full featured browser!)
The focus on the first release is good GuiControl integration for serving up the web in your Torque 3D application. I am also working on getting pages rendered on 3d surfaces (running into a slight issue with Material on this front, thanks to Tom for the IM session).
From there, better TorqueScript integration, picking on 3d sufaces (I actually, think this is best done in 2D and the novelty quickly wears thin, but it is probably worth having), performance tweaks (rendering on a separate thread, dirty rectangle), etc
I have some other (technically more important) work that needs doing with Torque, but this should definitely be a good starting spot and if anyone wants to dive in and help, that could be cool and I can try and help facilitate it.
#13
01/17/2010 (8:44 am)
Awesome work! Will there also be support for using webkit for the GUI? Like Overgroth is using it. 
#14
01/17/2010 (10:44 am)
Wow ! Really cool ! Great work ! 
#15
Looking forward to replacing it with the browser you found.
Has anyone found any real documentation on the inner workings of the texture and material systems?
01/17/2010 (10:39 pm)
Very cool!Looking forward to replacing it with the browser you found.
Has anyone found any real documentation on the inner workings of the texture and material systems?
#16
Again, this is way more than expected and I'm just glad it's coming out in any form at all. So, a huge thank you to Josh for getting this out to the community.
01/18/2010 (2:53 am)
@Tom: I'm just being feature-greedy as usual. The thought of having to update my web programming skills to the 21st century to build complex displays doesn't thrill me, but I suppose you're right about the power of this kind of solution making the GUI option kind of pointless.Again, this is way more than expected and I'm just glad it's coming out in any form at all. So, a huge thank you to Josh for getting this out to the community.
#17
Flash Work?
I know T3D web publishing enable Javascript <> Torquescript communication, so with your new tech, can it achieve Flash(Actionscript> <> Torquescript communication?
If it is possible then it would be huge imo, I always wanted to build game gui with Flash.
01/18/2010 (5:19 am)
@JoshFlash Work?
I know T3D web publishing enable Javascript <> Torquescript communication, so with your new tech, can it achieve Flash(Actionscript> <> Torquescript communication?
If it is possible then it would be huge imo, I always wanted to build game gui with Flash.
#18
It did become clear that for performance, I really need to get WebKit rendering on a separate thread... so putting that together now.
@Sei: Yes, Flash renders just fine. The input also was working in RENDER_NATIVE/RENDER_POPUP modes. I'll have to see what happens now with all the WebKit stuff on its own thread. It should continue to work with some sweat... probably going to have to leave RENDER_TEXTURE Flash input support for another day (we'll see).
For some ideas, you can check out a video of experimentation from last year using Awesomium... in old school Stronghold :)
01/18/2010 (10:54 am)
I'm currently working on the 3d rendering stuff (basic/advanced lighting). There was some good progress on this yesterday, web content with advanced lighting is really neat :)It did become clear that for performance, I really need to get WebKit rendering on a separate thread... so putting that together now.
@Sei: Yes, Flash renders just fine. The input also was working in RENDER_NATIVE/RENDER_POPUP modes. I'll have to see what happens now with all the WebKit stuff on its own thread. It should continue to work with some sweat... probably going to have to leave RENDER_TEXTURE Flash input support for another day (we'll see).
For some ideas, you can check out a video of experimentation from last year using Awesomium... in old school Stronghold :)
#20
1) All web content rendering/updating is now handled on a separate thread/CPU core. In the case of RENDER_NATIVE/RENDER_POPUP, the web stuff never touches the main thread. For RENDER_TEXTURE there is a quick mutex lock to transfer a cached memory buffer.
2) I spent a day fixated on getting Flash keyboard/mouse working in RENDER_TEXTURE mode, but I believe this was worth it. Being able to debug into Qt and WebKit is *really* nice for this low level stuff. A couple *minor* patches are necessary to WebKit in order for it to work... needle in a haystack but easy to apply now :)
3) Basic "dirty rectangle" and caching support with partial texture uploads. I also setup variable rate updates for animated content.
4) MatTextureTarget support for rendering web content onto 3d models using standard Material definitions and mapTo (Thanks to Tom for pointing me in the right direction)
So, that is most of the time consuming fiddly bits... a little "behind" doing this right. The fun stuff will be in exposing all the power of http://qt.nokia.com/doc/4.6/qtwebkit.html to Torque :)
- Josh
01/20/2010 (12:15 am)
A few updates:1) All web content rendering/updating is now handled on a separate thread/CPU core. In the case of RENDER_NATIVE/RENDER_POPUP, the web stuff never touches the main thread. For RENDER_TEXTURE there is a quick mutex lock to transfer a cached memory buffer.
2) I spent a day fixated on getting Flash keyboard/mouse working in RENDER_TEXTURE mode, but I believe this was worth it. Being able to debug into Qt and WebKit is *really* nice for this low level stuff. A couple *minor* patches are necessary to WebKit in order for it to work... needle in a haystack but easy to apply now :)
3) Basic "dirty rectangle" and caching support with partial texture uploads. I also setup variable rate updates for animated content.
4) MatTextureTarget support for rendering web content onto 3d models using standard Material definitions and mapTo (Thanks to Tom for pointing me in the right direction)
So, that is most of the time consuming fiddly bits... a little "behind" doing this right. The fun stuff will be in exposing all the power of http://qt.nokia.com/doc/4.6/qtwebkit.html to Torque :)
- Josh
Associate Fyodor -bank- Osokin
Dedicated Logic
I wanted to play around with QtWebKit until I found that you are already on it :) So, thanks a lot. This all looks impressive and I'm sure it will give developers a lot of ideas on how to make the game even more exciting.