Torque 3D Community Edition Discussion Thread

#21 I actually miss OpenGL...

Things like OpenGL, upgrading DirextX, and 64bit Torque could certainly be long range goals of the CE, but console specific goals are something for an individual team/studio to deal with.

#22 Since I've seen it mentioned a couple times now I just want to throw out a reminder that this community build is still bound by the EULA. Fixing the OpenGL layer does not grant the ability to deploy to consoles that use it, that is a separate license that you will need to contact licensing about.

Edit:
Heh, Michael beat me to it. Damn my verbose typing. ;)

#23 Thanks Scott. I made sure to mention it, but am glad to see an employee affirm my statement.

#24 If we want to develop the engine to target next gen games I think the most sensible thing would be upgrading DX9 to DX11 rather than develop openGL from the ground up, as most of the code is there it just has to be 'redefined', written in another way that is.
But it seems like a smaller task than developing openGL for targeting mac and linux. And for the ones who wants to develop to consoles although I believe it does take even more modifications to employ it to consoles as they have all modified their openGL version.
DX11 is not an impossible task, but a big task. I would make an entry point for it myself, but unfortunately I don't have either the knowledge or the time to take on this task atm.

#25 @Scott and Michael, thanks for clearing that up.
Sounds like a 64-bit T3D and a DX11 would be the way to go. Has garagegames though about re-implementing OpenGL?
I know Lukas has done some nice stuff with the particle system. Personally I would love to see a re write of the current particle system into something more modern to next gen.

#26 I think both OpenGL for *nix and the DX11 layer are good ideas.

#27 I would like to be able to deploy to modest hardware and still have features for the high end. Deploying for 64-bit would be fine, but I don't see a reason why you could not target 32-bit as well. Currently I don't see a whole lot of performance difference between the 64 and 32 bit systems other than RAM access above 4GB.

I would say if someone wants to upgrade the engine then fork it for high end spec machines that would be awesome. I would still like to deploy on lower spec machines. Ideally I would like to deploy on the lowest end machines that GG says T3D will run on.

If there is a way to combine the ability for low and high end that would be ideal for deployment. One feature I use with Half Life is the ability to select the version of DirectX. Making it a command line selection. It may force the engine to be more modular, but I think that might be a good thing.

Another performance gain might be to replace TS with a more advanced and possibly even JIT compiled version. Ooh, how hard would it be to JIT compile TS? With the amount of JIT work that could significantly increase the execution speeds. That would be immensely helpful for AI and even doing some simulation in the TS. Even just a subset of commands would be helpful. It is already in a byte code.

#28 Here is a possibility:
code.google.com/p/asmjit/
When I get time I will sit down and look at this. This may be a good complement to be able to compile specific functions and/or the entire TS code base.

#29 Okay, I dug around a bit to try and understand JIT stuff. So far I have found some interesting information about how JIT works. What I found that really surprised me was that there are ways to optimize the existing VM by changing how the opcodes are parsed. Here is a link to an article that talks a little about this:
wknight8111.blogspot.com/2009/06/understanding-opcode-dispatch.html

Here is the really interesting part of this article:

Quote:A subroutine-threaded core stores each opcode as a separate C-level function. Each op in sequence is called and then the op returns back to the runcore. This is two branch instructions to dispatch each op, compared to only one for a direct-threaded core. However, recent benchmarks I have seen in Parrot show that the subroutine core actually performs faster then the direct-threaded core does. This is because modern microprocessors have lots of hardware dedicated to predicting and optimizing control flow in call/return situations, because that is one of the most common idioms in modern software. This is a nonintuitive situation where more machine code instructions actually execute faster then fewer instructions.

Basically because modern processors are designed to maximize execution speed for function calls (branch optimization) calling functions within the switch statement inside a VM can actually make it faster rather than slower.

So without messing with JIT at all we could make the VM faster by replacing the code inside the each case statement in the switch of the CodeBlock::exec function in compileEval.cpp. This supposedly would make it faster because the CPU can better optimize the code execution and predict the next set of instructions and thus keep the CPU cache full. That is the theory anyway.

#30 Hmm I am interested in how much that would speed it up, actually I have no idea of how slow TS is never benchmarked it as I never got to really create a game.
Although I like the idea of a C++ scripting as it might be more intuitive to work with the same language in both scripting and coding.
But if fixing the VM would make TS as fast as scripting JIT in c++ that might be worth it. Also would that improve effectiveness of the eval function in TS?

#31 Things that would be nice for T3D to address. This is a list is based on improvements, resources from around the forum, and ideas that would be nice for T3D:CE to cover:

improvements
*SSAO: improve the current one
*DOF: improve the current one
*motion blur: improve the current one
*64-bit; layer for 64 bit HDR*
*subsurface scattering (skin): should be an option under the default.
*Parallax Occult Mapping: For all objects (player, static, terrain, etc)
*wetness: improve the one marcus L started
*camera system: a new system that ha Track,New Static,First Person,Third Person,Third Person Track,Third Person O-T-S,God View, New Orbit already pre-set.
*new GUI functions: Allow for new effects, for splash, button, and the HUD.
* lightning bolt: allow the ability to change the image


some resources that would be nice and help (request):

www.garagegames.com/community/blogs/view/5471
adv camera

www.garagegames.com/community/resources/view/21392
Recast NavMesh



ideas
1. AI Editor - it would be nice if T3D finally had an AI editor; using different resources exposing the behaviors of the bot, with the ability to create new classes with the right programming.

2.Cut scene/scene event Editor - Similar to the GMK it would have pre-set script events. Example walk in a room, garage door is trigger to raise across the room. Also the ability to create cut scenes.

Some other things that would be nice, is:
Screen-space global illumination- the ability to turn it on of off under the lighting in world editor.

Script driven weather system - it would use the cloud, precipitation, and Post FX classes to create varies weather events.

Physics, car (like GTA 4) and possible an APEX layer.

#32 @Kory already added a ticket for the SSAO improvement and a resource for this.
Also that SSAO resource opens up for Screen-space global illumination:
See the ticket here: https://lab.collateral-studios.eu/trac/ticket/19

I am using the Recast Navmesh resource atm (Although would love to upgrade to RecastT3D but financial issues keeps it from me) and it is very easy to integrate and works perfectly.
Might also be able to supply with some basic AI which uses this recast navmesh, but the AI is no wonder. I haven't focused on that developing it so it doesn't have alot of features.
Edit: Btw is there a stock motion blur? Where??

#33 @Lukas Yes I seen that, I'm just expressing the fact that I share your same interest in that (and a few other things). I would have the particle system on that list but you've but doing some great stuff and I also seen that you where looking into using them on a mesh and that would be awesome.
Also Volmetric clouds and smoke would be nice too!
Motion blur, well maybe not then it should be a request to have motion bur added.

#34 Also, a more site related topic. I would like to have a place where I could request help for the improvements I am working on. Right now I am using a request ticket in the ticket system but I think it would be nice to get it seperated from tickets, and asking on the forum gets the thread get lost between the other threads! :)
Having it on the community website would be nice as it is always on the list. It's also nice to have a specific place you could turn to when you want to help out fellow CE coders.

#35 Another thing:
Implementing SMAA instead of MLAA could be awesome, might get rid of that annoying flicker you see on a lot of your scenes.
Look this video for what I mean:
http://vimeo.com/31247769

MLAA website:
http://www.iryoku.com/mlaa/

SMAA website:
http://www.iryoku.com/smaa/

#36 @Lukas,
From what I understand a VM can spend a lot of time just jumping to the next instruction:

ip = 0;
for(;;){
  opcode = opcodebuffer[ip];
  switch(opcode){ // 1st jump to opcode label
    case OPCODE1:
      ...
      ip += 2;    
      break;  // jump back to switch instruction
    case OPCODE2:
      ...
      ip += 1;
      break; // jump back to switch instruction
    case OPCODE3:
      ...
      ip += 5;
      break; // jump back to switch instruction
    case OPCODE4:
      goto execFinished;
  }
}
execFinished:  // label to finish VM code execution

This does not look like a lot. However, when you consider it might just be loading a variable or doing a logic comparison for a particular opcode it can add up significantly. To the point where it starts to affect performance.

What I have seen to rectify this is stuff like this:

static int ip;

void (*funcList[])() = {opcode1,opcode2,opcode3,opcode4};

void opcode1(){
  ...
  ip += 2; 
  funcList[opcodebuffer[ip]]();
}
void opcode2(){
  ...
  ip += 1;
  funcList[opcodebuffer[ip]]();
}
void opcode3(){
  ...
  ip += 5;
  funcList[opcodebuffer[ip]]();
}
void opcode4(){
  ...
  return;  // returns back to exec
}

void exec(){
  ip = 0;
  funcList[opcodebuffer[ip]]();
}

Now this may look like it would be recursive and overflow the stack. But there is a compiler optimization that most modern compilers now support called: "tail call optimization". Apparently if the compiler sees that it will call a function at the tail end of a function it will not cause it to jump back to the end of the current function, but to the place where that function was called from. So with the second example it flows like this:
// given this list of opcodes
opcodebuffer[] = {0,2,2,1,2,1,3};

1. call from exec Jump to opcode1
2. Jump opcode3
3. Jump opcode3
4. Jump opcode2
5. Jump opcode3
6. Jump opcode2
7. Jump opcode4
8. return to point in call from exec after first call

With the switch there are twice as many jumps as with taking advantage of "tail call optimization". Also, I read something about processors being better able to predict function calls rather than near jumps and have been optimized to that effect. So even though the produced code is longer than a switch statement would be it is actually able to keep the pipeline fuller and do less lookup or cache invalidation. There are supposed to be further optimizations to this method, but it relies on specific compiler extensions that are only available on the GCC compiler.

#37 I think what would really benefit the engine, and what GG might not necessarily want to focus on, is support for basic gameplay features like doors and ladders. There is currently no best way to do these things, which is a huge impediment to people trying to make even simple games. I guess they're a little FPS-centric, though third-person shooters and any sort of adventure game could use them as well. After that, even things like moving platforms (elevators, anyone?) would be fantastic.

I'd also like to see the Player class improved a lot. Things like IK for leg placement on terrain would be great, and a better collision model than the current axis-oriented box (hopefully without needing a fully-fledged physics engine behind it).

I feel a little awkward posting here since I'm not actually on board yet... I fully intend to be, though, as soon as I purchase 1.2 this winter! (Southern hemisphere winter...)

#38 Well, what I've been doing in regards to player stuff which may be a bit weird, is I actually just exposed processTick callbacks for all shapebase objects(toggleable if it does it or not in the datablock) and updateMove for the player class specifically(again, toggleable via DB).

From there, you just need a few simple hook variables exposed to script and applies itself to stuff in the engine on those functions. In my case, I had a pitch and yaw modifier, and a move modifier.

Since it's much, much easier to handle gameplay-specific stuff, or stuff that is actively affected by current gameplay via script, you can leave the heavy lifting code in the engine, and gameplay code in script.

To use Dan's example of ladders, in my case all I needed to do was when I interacted with a ladder object, in script I set that I was in a 'ladder climb' type state, and in the script update move I can 0 the x and y movement, and translate any current movement to Z, allowing me to climb up or down the ladder.

It's very fast in my experience, as long as you don't try to start doing stuff like physics and whatnot, and it's way easier to code gameplay stuff than having to hop into the engine every time. It's an easy implementation, and I think it'd be worthwhile to add that sort of thing in there. It's SO much easier to add regular 'tropes' of shooters and other games like this(script-only gun sway? check. movement speed affected by health? check. ladders, climbing, etc? check.) And you can just have it not do it if a certain player type shouldn't have the ability, which would be handleable via the datablock being the callback point.

Just my 2 cents on that subject(or, looking at my comment, my dollar)

#39 @Jeff that is a really cool idea. Having some hooks like that can help with creating different gameplay features.

You just reminded me of something I would definitely like to see in the engine. Back in TGE days I had explored building a modular class system that would basically allow you to build up an object by including different features. The features would be things like: networking, phyics type (player, vehicle, physics plugin), mesh type (soup, static, custom, etc), animation (bone, algorithmic, custom), hitbox (simple, segmented, custom, etc), and any other feature that could be componetized. I am not sure how to build such an object in C++. Maybe multiple inheritance.

My thought process on making it modular is that you can completely test each part independently and get the code to be rock solid. Then each part can be included into one object and at that point you have to design the interfaces, but you are not troubleshooting if you put the right code in the right place. This is especially a pain with networking code. There were a few times that I spent hours trying to find a networking issue only to eventually give up on figuring it out. If the code were more modular then this might help with the learning curve and being able to use the engine effectively. Right now the Player is very monolithic. At least it was last time I looked (again TGE days). I seem to remember lots of inter-dependencies.

I am not sure how to structure it, but for networking it would be great to have some sort of assert check. So when you set something in the stream the code at compile time will detect if you set it in the read part of the stream. That kind of checking could save someone hours, days, even weeks of frustration and guess work.

#40 All those are good suggestions, the gameplay features would be a nice feature to add to the scene event editor. It can have a list of pre-set gameplay related function. For example, ladder climbing, or walking in a room and a trigger opening a door, etc. All the function can be resources port to 1.2 that would make for great gameplay features.