Torque 3D Workflow for Modular Level Design

#1 I've done some testing using a single datablock and the StaticShapeData datablock works well! I'm still trying to work out how to apply different materials to a single blank mesh datablock, but if push comes to shove creating a datablock with each material is totally feasible.

Basically, just creating a StaticShapeData datablock allows me to conjure up new tiles with just a few lines of script. If an entire room uses the same floor tile, it's just calling to the same datablock. Good stuff!

Performance
There are 2 options I am exploring:

• Assembling in modeling app and exporting the whole as a multi-meshed level object.
• Creating datablocks for each piece and assembling in Torque World Editor.

It has yet to be seen which will deliver the best performance. Here are my initial observations:

If this is assembled in the World Editor using datablocks, collison shapes can be fit per tile to fit by 'bounds' or convex hulls depending on the tile's complexity. Alternatively, if this is assembled in my 3D app I'd end up creating separate collision meshes to span an entire floor rather than 'per tile'. Obviously the latter is using one huge collision shape as opposed to several smaller ones...although I'm not sure if it's really going to matter in Torque performance wise.

Ultimately I believe it will come down to either creating datablocks per tile or by larger prefab components of the level. In either case, I suppose it would be safe to just not use Torque's bounds or convex hulls collision at all and just add collision shapes in separately.

#2 For once, a solution has been found on swift wings. With just a couple small tests, it did not take long to realize I have answered my own question:

Yes, pound for pound, we are asking less of the engine if we set all this up in our modeling app beforehand.

A handful of datablocks, or even potentially a single datablock containing several meshes is much better than calling on datablocks per tile no? Potentially avoiding 100s, if not 1000s, of calls to tile datablocks.

Unless less is more, meaning that there is some advantage to having levels call datablocks 'per tile', I believe I'll just build in Blender.

#3 Unless you're making a new datablock for every single tile, I suspect the overhead comes from having more objects, not more datablocks. As for changing materials, search for setSkinName (I think it's called) and go from there. It's not ideal, but it might have to do. I wish it were otherwise. At the moment, materials are associated with shape files, not with objects (is my simple understanding).

I'd do some digging for you if I had more time!

#4 No worries there Daniel, thanks for the reply!

I'm fortunate enough (thanks to the community here) to have a fair understanding of TorqueScript and the import/export process. I've tested all of this now, and in the end the performance difference is negligible.

If you build it all with tile datablocks, you can use the setSkin stuff in script if you really wanted to build it that way.

I'm a sucker for extreme scrutiny, so I will end up just assembling it and importing in accompanying collision shapes. One initial test shows this works very well in Torque!

#5 TsStatic is what you want to use for non-dynamic objects (eg: buildings) which don't move or affect anything other than their collision being set at level startup.

StaticShape is what you use for semi-dynamic objects which have script callbacks (eg: add health or damage object onCollision) and as such have much more overhead.

If you want to create a whole level in a 3D modeling app, use multiple objects, export them independantly as Collada and each object will keep it's position in relation to origin (0,0,0). This is what I did for this level of an old and since scrapped project - that's 98 TsStatic objects exported, all retaining their position thanks to Collada's inheritence.




Here's what an abridged flythrough using lightmaps looks like.



You can also do it the other way by exporting individual models based around origin in your app and line them up manually in Torque. If you keep the bounding boxes the same size then it'll make them all square which makes things a lot easier when trying to write a script to fit them together.

#6 @Steve: You never cease to amaze me with the stuff you come up with. First off, that level design and implementation is sooooo solid! Nice work, that looks really good. Can't believe you've scrapped that after putting in so much work. I mean, I might update a texture or so but man it's nice!

Anyhow, thanks for the rundown on the shapes. I've used the StaticShape datablock to create a door that animates when the player is within range (sci-fi sliding door woot) and in doing so I had a thought: "What if I could just create a datablock and re-use that single datablock every place a level uses the same piece." As you've indicated, this will work but ultimately it's unnecessary. Creating the datablocks for larger sections of the level (which may include several other meshes) is definitely the way to go. Rather than having datablocks for all the small tiles.

You are 100% accurate about the Collada inheritance. I discovered this on my first initial test trying all of this out. It's really, really, cool stuff and extremely powerful. Thanks for your input Steve, at least I know I'm on the right track!

P.S. I've experimented with lightmaps in Torque, and out the box Torque is pretty strong in this department. A pulsing blue light with a blue-ish hued lightmap texture makes for a really good looking glowing 'power panel'.

#7 I wanted to report here with some fantastic results. To start with, I have crafted a 'chunk' of the level that in itself is fairly large. It is comprised of a little over 200 meshes, and this massive twist of corridors, platforms, and sci-fi-see-through-window-goodness is merely the first corner of the level. Sounds as if the task of getting that into Torque without lagging all across town and back would be kinda rough right...?

Well, yes and no, really. It's taken an understanding of 2 main concepts to get this all into Torque running smooth:
Datablocks
3D Modeling
Armed with this knowledge, it's all go from there. Here's how I'm going:

• Assemble modular mesh pieces in Blender (build the level). No props, just level geometry.
• Subdivide the level into 'chunks'. In my case I have 11 chunks. It's a good idea to divide these chunks in ways that u can re-use them.
• Export each chunk as a separate .DAE file. As Steve has indicated earlier, just export the chunks out and don't move them. On import into Torque they will retain their positioning data.
• So then it's just setting up my hierarchy for the mats and shapes, and importing the level chunks.
• Of extremely great importance is if a material is re-used in Blender to be sure all cases are targeting the same material. Re-creating various materials for different shapes that may share the same texture = bad news for Torque. If a wall uses the same texture as a floor someplace else, just be sure they are both named 'floorA_mat' for example.
• Setting up mats and collision in Torque is a breeze at this point. Much to my surprise, after I imported all of these 11 chunks I simply enabled the collision 'by visible mesh' within Torque and honestly didn't take any hit*(see below).
• At this point, it's time to duplicate and expand on the level since all of this giant work is just one quadrant of the whole thing. Now just create datablocks for the chunks and it's a done deal. Depending on the level design, you may or may not want to use datablocks for some of the chunks. If it's a big section that you will likely use like maybe a few times in this level only, maybe just drop a few TSStatics in. If it's a corridor you might end up tiling a lot, u may want a datablock of that piece so you're only calling to the single datablock.

Following the above, I am traveling around in my level with the FPS metric reading between 170+ - 190+ FPS on average.

Notes:
This is a first pass, I haven't added any extra props (lights, ceiling pipes, etc.). Although all of the textures are using normal and specular maps so far.

I'd also expect to be able to optimize performance even further by disabling Torque's 'built-in' collision and creating interior collisions manually. The logic being if I'm providing simpler mesh geometry, I'm asking less of the engine (not having it create collisions for the 'hulls').

*I've been schooled by AndrewMac a little bit about fps being a poor measurement of performance. I respect this fact, although it is the quickest and most visual way for me to personally monitor these tests. Totally open for suggestions on ways I can improve my tests to ensure my level-building methods are indeed gauged off a good performance metric.

#8 Just saw this post Jesse. I'm doing this the same way for our level design. I'm creating a city but am doing this in the modeling application. I'm them exporting the city in sections and by individual building from the modeling package using the origin positions. I find this more effective performance wise but also building the level is more interactive and detailed. It also avoids a lot of the z fighting issues that arise and you can hide a considerable amount of polys from sight thus facading a lot. Here's a quick building and pavement sections exported as different items to torque. Still very much WIP but give the idea.


Doing it this way also enables some nice control on the LODs too but also enables you to blend texture detail beteeen the models in the modeling package so they look more natural when put together in the engine.

Good luck with the level builds.

#9 Standard collision meshes in Torque are supposed to be convex only. Torque's "visible" collision is an exception to this. I bring it up because it occurs to me that you could make your collision meshes as if they were part of the regular "visible" mesh, but with a transparent material. Then set collision on that mesh instead of your buildings/streets.

My concern with this approach is blending the transparent material. I don't know if there will be a performance hit, but I think not because of another thread where someone mentioned that the renderer skips blending if alpha is 0....

#10 Jesse - pics or it didn't happen ;). Seriously, I wish I was that efficient at assembling stuff in 3D apps. Glad to hear it worked so well for you.

#11 @Tim: Thanks for the confirmation of your successes using this method. It does seem to work really well, as opposed to just trying to plug in each modular 'tile' individually. Tbh, my level 'chunks' currently contain sub-meshes within them. Honestly, given a little extra time and attention to detail I could merge all of those and then remap the UV of the whole thing...but it's a bit of extra work. With most of the sub-meshes already sharing textures in many cases, it may not be worth the effort.

@Richard: What I plan to do with the collisions is really simple. I'm just going to open up each prefab 'chunk' in Blender and select only the interior faces. Then I'll just select the inverse, delete it all, and be left there with only the inner faces for my collision mesh. Next I'll just not apply any material at all! Torque translates this as invisible :)

@Danny: Yes, of course. I've been tossing out quite a few claims of my successes on multiple fronts...but you guys haven't seen a thing have you? My apologies, I'm off to get a few pics to share :D

#12 Okay, I took a few quick shots of the test level's 'NE Quadrant'. In total, it's a bit over 200meshes assembled and then subdivided into 13 (lucky?) 'chunks'. Each 'chunk' is comprised of other sub-mesh components of the modular kit. Again I emphasize that in many cases, these sub-meshes share the same textures and your materials should be set up accordingly both in Blender and in Torque.

I've taken a range of shots, varying from 'best case' to 'worst case' to give an idea of what I'm looking at:

TestA:

TestB:

TestC:

TestD:

TestE:

Additional Notes:

• 1440 x 900 resolution
• No post-effects
• No props
• Advanced lighting with no added level lights
• This test yielded an average of 168fps, with no optimization.

As you can see from the shots, the peaks in performance will occur when facing outward from the rest of the geometry. Additionally, it's very noticeable that when I'm in a long corridor, my FPS should be much lower...indicating that there isn't any culling taking place of the geometry behind the corridor walls? I believe that if there was some sort of backface culling taking place, this would run leaps and bounds better. The 'mesh-culling' checkbox option in Torque doesn't seem to affect anything but I'm not really sure how to go about testing it.

#13 Jesse. Looks good. Doing it this way also enables you to limit the 'look' of the modular approach too. With doing it in the engine You have to use the same static mesh. If you do it in your 3d package you can use all the modular sections then adjust them to avoid the modular look. Use the modular sections like a kit and then adjust them all slightly. The building above is made this way. The top floor where the windows are is made up of s ix sections joined together in the 3d package and then I tweak the angles etc so they don't look so uniform. I suggest the same. This way is quick and more natural than sticking loads of stuff together in the engine. I like your modular style. Iv suggest doing the same and just tweaking things to make them look less modular in the model.

Great stuff fella!

#14 If you want culling you'll have to use zones and portals - there's also some blocker object but I can't remember the name of it off the top of my head.

Lighting creates more overhead, especially on a huge map with many lights. I created groups for my lights and turned them on and off as needed.

Quote:
Can't believe you've scrapped that after putting in so much work. I mean, I might update a texture or so but man it's nice!

Project was too large and I couldn't do it to the level of finish I wanted ... plus I stopped playing heavily scripted FPSes.

If you do fancy a closer look, the demo is still up, has 2 levels, one story mode and 2 using procedural enemies.

yorks.deta.in/download/YR_demo2_setup.exe

#15 Occlusion volumes, but I would experiment with them a bit - I remember some issues where shadows of objects you could not see but should be able to see the shadows of were not being rendered once upon a time....

#16 @Everyone: Thanks for the advice. Looks like I've got a bit more to study to get this all working how I want. Guess I'll look more into setting up zones and portals first, I've seen mention of this in a few places already. I can imagine this may get a little tricky with all of the windows in this level.

@Steve: Just downloaded that demo, thanks! I bet I learn a thing or two from it, what do you think? :D Much appreciated!

@Richard: Occlusion volumes? I haven't seen anything to do with that yet, any idea where I might start my search around this?

#17 They're in the World Editor Library, Level tab, Level group - along with Player Spawn Sphere, Zone, Zone Portal, etc. I think the idea was that you could guarantee things behind it would not be drawn - like an iron curtain of culling.

#18 Handing out thanks like candy here at this point, but again thanks man. That sounds exactly like what I'm looking for.

With a little attention to detail, I might just get this thing running smooth. I mean, don't get me wrong, I'm pretty happy with what I've got. But I know it's possible to do better, and I won't settle for anything less :D

#19 It turns out that there has been a lingering issue I was unaware of on mission cleanup. Also one of my server-side scripts was causing problems. Long story short, the performance is approx. 30fps better than tested previously at all times! On the above tests, I had only a single quadrant of the level. Today I duplicated that quad, and with double the level geometry tests are nearly on par with the above. Man, Torque rocks, it really does! It certainly takes a bit of learning, but it rocks!

*Note: About the occlusion volumes - It turns out in my case that the overhead associated with using the volumes outweighs any benefits from culling. In other words, performance actually decreases if I use the occlusion volumes. I still feel as if there is room for improvement in the culling department, but as of yet I've not found any sort of solution.

#20 Have you started adding zones and portals?