With the impending end of cpu speed increase, is there any plans for a threaded T3D?

#1 excellent question and I'm very interesting in this.

#2 T3D already use multi core cpu's. I'm not sure how it is balanced but when I did try to get it run on FreeBSD there was a lot of work to implement the multi core / multi cpu code ;) You can also see it on windows in the task manager that not only one core/cpu is used.

Edit: Threads are used since TGE for different jobs. As far I remember right the telnet console runs in an extra thread.

#3 T3D may not be fully optimized for multi CPU's, but multi core CPUs, modern motherboard chipsets and operating systems as old as XP are designed to optimize the multi CPU usage.

At around the 4GHZ mark, the problem becomes keeping the CPU consistently fed with data. CPUs excessive of 4GHZ is not rare or difficult, it is RAM speed(IO BUS) that needs to evolve before a 4+GHZ CPU is efficiently feasible.

#4 Yes it uses multi-threads farming off stuff like SFX, theora and others bits from what I can remember but I thought most of the heavy weight stuff (rendering, physics,etc) all runs on one thread?

#5 Well the example i have in mind is of Armed assault which is to my knowledge the only REALLY multi threaded game out there, it's engine has the following threads:

Rendering
Models loading
Texture loading
File operations
AI

I heard intel announcing somewhere that they will keep piling up cores in machines, so maybe 10, 50, 100? core machines are up ahead, maybe not now, or in one year, but if parallelism is gonna be the only new way up i guess it's better if we prepare as soon as possible.

#6 Massively multicore systems are the future and Torque will support them very well.

#7 Oh got any more news on that Tom? plans or things under way already?

#8 Actually Intel announced a few weeks back that they are accelerating their road-map and that 24-32 core CPUs are closer than you think. 2-3 years out.

#9 @Tom:
Awesome!!!

#10 There isn't any specifics i can share, but rest assured that the team is well aware that Torque has to scale across many cores in future releases... and not just for physics and sound.

#11 Even if I have no details, I'm really pleased to ear such news. That will fit perfectly with my needs :D

#12 Yeah it's great news indeed :)

#13 I have been thinking about threading some parts of Torque as a side project. Moving the entire thing to a be fully thread scalable is not possible without some major re-writes (I'm looking at you, TorqueScript and ProcessList), but some of the techniques Valve used to multithread Source engine can be applied.

I'm thinking of the "divide and conquer" approach. It works like this: When an application needs to perform some operation of a large block of data, split the data into smaller chunks and have many threads consume these chunks in parallel, and when they're all done the main thread can continue.

A prime candidate for this is character skinning: on a quad core, have four threads skin 1/4th of the vertices in parallel. In theory it would make skinning 400% faster on a quad code and 200% faster on dual cores.

Other functions that could be parallelized off the top of my head are: triangle ray casting, collision detection and polygon clipping (decals).

I also have been thinking about ways of adding asynchronous loading support to the resource manager, but that'd be a bit more involved.

BTW, Torque already features some very, very nice support for creating threaded stuff. The ThreadSafeDeque template is amazing for passing data between threads.

#14 Intriguing Manoel, wish i was a more clever programmer...


Everyone else, take special notice that Tom never actually says anything about Torque optimizations for muticore CPU. Stop being spoonfeed fan boys and think for your selfs, its embarrassing to the cognitive function that defines the human race apart from a hive of ants.

Quote:Massively multicore systems are the future and Torque will support them very well.

Is the exact same as saying "Torque will support Massively multicore systems very well."; what it already is doing.

#15 The cool thing is that once T3D has DX11 it can actually start to also do the rendering related stuff in multiple threads :) (and as we all know, the drawcall border is one of those where you tend to hit the wall first aside of fillrate and VRAM kills. Though the instancing will naturally do a good job against the drawcall problem too)

#16 I had thought DirectX variation of multi threading was a product and work derived from supported video drivers, not the DX code itself, and one would benefit from it simply by having compatible hardware and drivers. But i have only tentative research about subject.

#17 @Manoel

Quote:Moving the entire thing to a be fully thread scalable is not possible without some major re-writes

We're doing both system refactors and tasks/jobs to get Torque to scale on multicore systems.

Quote:A prime candidate for this is character skinning

Specifically with character skinning both Pat Wilson and I have implemented a threaded version, but somehow we didn't end up getting that into Torque 3D so far. If you have lots of skinned characters its a life saver and the only way to make it perform well. I'll bring it up with Matt and see if we can get an implementation in there soon.

Quote:Other functions that could be parallelized off the top of my head are: triangle ray casting, collision detection and polygon clipping (decals).

I think what you'll see from Torque in the future is that we'll depend on the physics system for all ray casting and collision. PhysX is very scalable in both its collision work and ray cast work.

Not sure what will happen with decals yet, but i don't think multicore will solve the problem in the long term. Decals belong on the GPU.

Quote:I also have been thinking about ways of adding asynchronous loading support to the resource manager

So have we... its not a simple thing to do. You have the rework all of the resource loading functions and break it up into CPU work and GPU allocations.

@Caylo

Quote:Tom never actually says anything about Torque optimizations for muticore CPU.

We are doing optimizations to Torque for multicore CPUs. ;)

@Marc

Quote:once T3D has DX11

Its really high on our list, but work on the material system has to come first as we need SM 5.0 out of ShaderGen.

#18

Quote:@Caylo

Quote:Tom never actually says anything about Torque optimizations for muticore CPU.

We are doing optimizations to Torque for multicore CPUs. ;)

Tada! Now your making a statement worth excitement.

EDIT: also thanks for the inside view, the single most valuable way for curious developers to know what direction Torque related details are moving.

#19

Quote:
Specifically with character skinning both Pat Wilson and I have implemented a threaded version, but somehow we didn't end up getting that into Torque 3D so far. If you have lots of skinned characters its a life saver and the only way to make it perform well. I'll bring it up with Matt and see if we can get an implementation in there soon.

This interests me greatly: the project I'm working on has quite a few characters visible all the time and skinning is at the top of our CPU usage. I was already scheduling time to thread the skinning code, but if you guys have something even remotely usable it would be a godsend.

#20

Quote:if you guys have something even remotely usable it would be a godsend.

I'll do what i can to get it in there for 1.1 or release the code as a resource for you.