Torque 3D DirectX11 Port - DirectX 9 refactor
by Lopuska · in Torque 3D Professional · 03/11/2014 (6:23 pm) · 81 replies
Hi everybody!
I'm Lopuska, I'm from Italy.
I'm taking some of my time to try a conversion of Torque3D from DirectX9 to DirectX11.
This is my current status, there's still a lot of work to do...
I don't have many times in this period, if anyone would like to help me for this big task it would be great!
Before that, I've done a lot of code refactoring on D3D9 layer, also to make porting to D3D11 easier.
Here there are change list about DirectX9 refactoring:
- Better *.cpp organization / code cleaning
- Removed D3DX calls dependencies. (deprecated by Microsoft)
- Removed D3D9 XBOX360 old reference.
- Shader/Constant buffer refactor with a custom parser.
- gfxD3D9Shader total refactor.
- Mipmap autogen fixes
- Minor fixes under cubemap
- Speed improved on texture loading
- Removed 24bit format (deprecated since DX10) (i convert it to 32bit)
There is a better explanation about it on code as comment.
- Some performance improvement (especially under a lot of dynamic_cast used in wrong context)
- Removed fixed pipeline (deprecated by Microsoft since DX10)
- Many other minor bug fixed.
With this patch, you don't need DirectX SDK installed anymore. It's deprecated as well. You just need Visual Studio with its windows SDK.
Everything is inside d3d9.h/d3dcompiler.h -> d3d9.lib/d3dcompiler.lib
So, to try this patch, remove from linker input d3dx9.lib and dxerr.lib and DX_SDK includes to avoid conflicts and errors during compile/link.
(You can also remove d3d9.lib input linker because since this is Microsoft specific code, I've included it from #pragma directive inside *.cpp)
To remove fixed pipeline, I've used the same method used by Luis Anton Rebollo.
So if the community want to merge this part of code with Luis' work, it will be very easy because from high-level GFX layer you just need to make the same calls that Luis already do for OpenGL.
With the actual DirectX9 implementation, the situation should be:
//from precipitation.cpp
if(RenderDevice == OpenGL) //ugly!
GFX->setupGenericShaders(GFXDevice::GSTexture);
else if(RenderDevice == Direct3D9)
GFX->disableShaders();
With this Patch we have just:
GFX->setupGenericShaders(GFXDevice::GSTexture); //Now also DirectX use shaders without FFP
The main branch is:
https://github.com/Lopuska/Torque3D/tree/D3D9_D3D11_R%26D
For DirectX11 situation, I'll start a new thread.
byee!
Lopuska
I'm Lopuska, I'm from Italy.
I'm taking some of my time to try a conversion of Torque3D from DirectX9 to DirectX11.
This is my current status, there's still a lot of work to do...
I don't have many times in this period, if anyone would like to help me for this big task it would be great!
Before that, I've done a lot of code refactoring on D3D9 layer, also to make porting to D3D11 easier.
Here there are change list about DirectX9 refactoring:
- Better *.cpp organization / code cleaning
- Removed D3DX calls dependencies. (deprecated by Microsoft)
- Removed D3D9 XBOX360 old reference.
- Shader/Constant buffer refactor with a custom parser.
- gfxD3D9Shader total refactor.
- Mipmap autogen fixes
- Minor fixes under cubemap
- Speed improved on texture loading
- Removed 24bit format (deprecated since DX10) (i convert it to 32bit)
There is a better explanation about it on code as comment.
- Some performance improvement (especially under a lot of dynamic_cast used in wrong context)
- Removed fixed pipeline (deprecated by Microsoft since DX10)
- Many other minor bug fixed.
With this patch, you don't need DirectX SDK installed anymore. It's deprecated as well. You just need Visual Studio with its windows SDK.
Everything is inside d3d9.h/d3dcompiler.h -> d3d9.lib/d3dcompiler.lib
So, to try this patch, remove from linker input d3dx9.lib and dxerr.lib and DX_SDK includes to avoid conflicts and errors during compile/link.
(You can also remove d3d9.lib input linker because since this is Microsoft specific code, I've included it from #pragma directive inside *.cpp)
To remove fixed pipeline, I've used the same method used by Luis Anton Rebollo.
So if the community want to merge this part of code with Luis' work, it will be very easy because from high-level GFX layer you just need to make the same calls that Luis already do for OpenGL.
With the actual DirectX9 implementation, the situation should be:
//from precipitation.cpp
if(RenderDevice == OpenGL) //ugly!
GFX->setupGenericShaders(GFXDevice::GSTexture);
else if(RenderDevice == Direct3D9)
GFX->disableShaders();
With this Patch we have just:
GFX->setupGenericShaders(GFXDevice::GSTexture); //Now also DirectX use shaders without FFP
The main branch is:
https://github.com/Lopuska/Torque3D/tree/D3D9_D3D11_R%26D
For DirectX11 situation, I'll start a new thread.
byee!
Lopuska
#42
The solution in this case would be to calculate an appropriate row count and row size from the source texture and use that when copying.
Apart from that looks neat!
03/21/2014 (5:22 am)
Anis, I had a look at your code for reference in the interest of removing D3DX but I noticed a problem with your cube map loader (in the github repo). In void GFXD3D9Cubemap::initStatic(GFXTexHandle *faces) you lock the rectangle and do a row-by-row copy, however if the cube map is being created from a set of DXT-compressed textures this will fail miserably as the row count won't equal the height of the image.The solution in this case would be to calculate an appropriate row count and row size from the source texture and use that when copying.
Apart from that looks neat!
#43
Thank you really much for your report.
Don't worry about this bug. All this part has been totally revisioned.
Now the update is not bit per bit, but it made directly from driver with StretchRect and UpdateSurface.
I copy-past here what i have already said in the other thread just to inform all:
This evening I'll publish last patch.
Forgot WindowsXP. It's not compatible, because my refactor is based on the DirectX9Ex. Sorry if someone use it :(
You need to have Windows Vista or +.
http://www.microsoft.com/en-us/windows/enterprise/end-of-support.aspx
this is for people who know what is DirectX9Ex, basically it's a update to make DirectX9 works better on the last Microsoft's operating systems:
http://msdn.microsoft.com/en-us/library/windows/desktop/ee890072(v=vs.85).aspx
Now I'll continue with DirectX11 work :)
Bye =)
03/21/2014 (7:10 am)
Hi James!Thank you really much for your report.
Don't worry about this bug. All this part has been totally revisioned.
Now the update is not bit per bit, but it made directly from driver with StretchRect and UpdateSurface.
I copy-past here what i have already said in the other thread just to inform all:
This evening I'll publish last patch.
Forgot WindowsXP. It's not compatible, because my refactor is based on the DirectX9Ex. Sorry if someone use it :(
You need to have Windows Vista or +.
http://www.microsoft.com/en-us/windows/enterprise/end-of-support.aspx
this is for people who know what is DirectX9Ex, basically it's a update to make DirectX9 works better on the last Microsoft's operating systems:
http://msdn.microsoft.com/en-us/library/windows/desktop/ee890072(v=vs.85).aspx
Now I'll continue with DirectX11 work :)
Bye =)
#44
Now you can test it =)
https://github.com/andr3wmac/Torque3D/tree/d3d9-refactor
WARNING:
Since DirectX 10+, the primitive topology D3DPT_TRIANGLEFAN no longer exists. This is a problem, because we need to found all "GFXTriangleFan"
reference and change all vertex order used before drawPrimitive call...
Does someone have any idea to remove triangle fan and use instead triangle strip/list?
It's the last thing we need to do on direct3d9 refactor.
OpenGL has still this topology format, Microsoft has decided to remove it just to formalize nvidia and ati hints...
So maybe I can talk with Luis to ask him if he want to remove triangle fan support also from his opengl layer.
If we don't do that, on high-level gfx logic, we need to branch every "pVert->point.set" for DirectX and OpenGL. It's better to abstract that and have this part unified.
03/23/2014 (11:27 am)
The last update has been pushed in the d3d9-refactor branch.Now you can test it =)
https://github.com/andr3wmac/Torque3D/tree/d3d9-refactor
WARNING:
Since DirectX 10+, the primitive topology D3DPT_TRIANGLEFAN no longer exists. This is a problem, because we need to found all "GFXTriangleFan"
reference and change all vertex order used before drawPrimitive call...
Does someone have any idea to remove triangle fan and use instead triangle strip/list?
It's the last thing we need to do on direct3d9 refactor.
OpenGL has still this topology format, Microsoft has decided to remove it just to formalize nvidia and ati hints...
So maybe I can talk with Luis to ask him if he want to remove triangle fan support also from his opengl layer.
If we don't do that, on high-level gfx logic, we need to branch every "pVert->point.set" for DirectX and OpenGL. It's better to abstract that and have this part unified.
#45
but just for shaders u might need to implement triangle list adjacent - for terrain shader and such
03/23/2014 (7:46 pm)
triangle list seems to work for most but just for shaders u might need to implement triangle list adjacent - for terrain shader and such
#46
Apart from that there's just a few things which use PrimBuilder and some which make very small triangle fans. Also for some of the debug shapes PrimBuilder makes large triangle fan lists.
However should be relatively simple to convert these to strips or lists of triangles.
03/24/2014 (8:56 am)
It should be noted that the DTS code makes use of GFXTriangleFan as shapes can include triangle fans in them. However setting smUseTriangles should convert these into normal triangle lists.Apart from that there's just a few things which use PrimBuilder and some which make very small triangle fans. Also for some of the debug shapes PrimBuilder makes large triangle fan lists.
However should be relatively simple to convert these to strips or lists of triangles.
#47
*Edit:
Oh just did a search through the source code, GFXTriangleFan is used an awful lot throughout T3D. This would certainly take some work to remove it.
03/25/2014 (6:30 am)
Both NVidia and Ati/Amd have been saying for years to only use triangle strips and lists and obviously Microsoft heeded this advice and removed triangle fans from their recent API's. Personally i think this is a good excuse to rid torque completely of triangle fans. The big GPU makers are not making this stuff up ;-)*Edit:
Oh just did a search through the source code, GFXTriangleFan is used an awful lot throughout T3D. This would certainly take some work to remove it.
#48
With:
?
03/25/2014 (9:12 am)
It looks like a lot of those are references are to PrimBuilder though. Unless I misunderstand PrimBuilder could we not just remove TriangleFan as an option and replace all instances of:PrimBuild::begin( GFXTriangleFan, 4 );
With:
PrimBuild::begin( GFXTriangleList, 4 );
?
#49
03/28/2014 (7:44 am)
I keep getting the error "cannot open 'x3daudio.lib'". I have the windows 8 sdk installed so im not sure whats happening here. Any input will help.
#50
03/28/2014 (8:50 am)
Easiest solution is to remove sfx/xaudio from the project. 
#51
and the like need to be
or they won't get converted to 0 in order to trigger that AssertFatal if the condition arises.
03/28/2014 (11:17 am)
Still going over this in conjunction with the beamng and dev branches, so could be out of date, but should note:AssertFatal(static_cast<GFXD3D11StateBlock*>(block), "Incorrect stateblock type for this device!");
and the like need to be
AssertFatal(dynamic_cast<GFXD3D11StateBlock*>(block), "Incorrect stateblock type for this device!");
or they won't get converted to 0 in order to trigger that AssertFatal if the condition arises.
#52
03/30/2014 (8:45 pm)
I tried doing that, cleaning the solution and then rebuilding and that error still comes up. Im using visual studio 2013 on windows 8 without the DX SDK.
#53
03/30/2014 (9:14 pm)
Under the project properties you need to remove x3daudio.lib from Linker -> Input -> Additional Dependencies. Also, remove: d3d9.lib, d3dx9.lib, and DxErr.lib.
#54
03/31/2014 (5:47 am)
yes, remove also x3daudio.lib in addition of Andrew's instruction =)
#56
I'll commit the new graphics code next week as agreed with Luis when he pushes his last updates!
(We are working together to make the gfx code and interface similar as much as possible beetween OGL and D3D)
NOTE:
When DirectX11 will be finished, it's probably going to totally replace D3D9.
DirectX9 is really old, and supporting it in 2014 makes it complicated...
Luis has suggested me to deprecate D3D9 and I totally agree with him.
Also, DirectX11 has feature level, so if you don't have a card that supports DirectX11, you can still use it under emulation mode having a DX9-DX10 graphics card.
04/05/2014 (8:41 am)
Triangle Fan has been removed and successfully replaced with TriangleStrip! :DI'll commit the new graphics code next week as agreed with Luis when he pushes his last updates!
(We are working together to make the gfx code and interface similar as much as possible beetween OGL and D3D)
NOTE:
When DirectX11 will be finished, it's probably going to totally replace D3D9.
DirectX9 is really old, and supporting it in 2014 makes it complicated...
Luis has suggested me to deprecate D3D9 and I totally agree with him.
Also, DirectX11 has feature level, so if you don't have a card that supports DirectX11, you can still use it under emulation mode having a DX9-DX10 graphics card.
#57
Assuming the above is true, does anyone have any benchmarks showing performance differences between T3D 3.5.1 vs. T3D+D3D9-refactor? Dropping XP support seems like a high cost to pay unless there are significant performance improvements.
04/05/2014 (9:05 am)
Am I correct in saying that the D3D9 refactor makes Vista the minimum requirement for T3D? Assuming the above is true, does anyone have any benchmarks showing performance differences between T3D 3.5.1 vs. T3D+D3D9-refactor? Dropping XP support seems like a high cost to pay unless there are significant performance improvements.
#58
04/05/2014 (9:10 am)
Very nice keep up the great work.
#59
Windows XP is over 12 years old. Microsoft has extended support for it longer already but it's finally going on the scrap heap (this week actually, as it happens).
From next week onwards, there will be no security patches or updates for XP and most other software vendors will also drop support for it. Most new software probably won't even run on it.
I'd be very surprised if anyone with a PC running XP has hardware good enough to play modern games anyway - and if they do, they can easily upgrade to a newer operating system.
I think it makes absolutely no sense to try to support multiple different DirectX codebases when almost all modern PCs will run DirectX 11 quite happily and there's a working OpenGL option as a fallback anyway.
04/05/2014 (11:22 am)
Quote:
@Chris Haigler
Dropping XP support seems like a high cost to pay unless there are significant performance improvements.
Windows XP is over 12 years old. Microsoft has extended support for it longer already but it's finally going on the scrap heap (this week actually, as it happens).
From next week onwards, there will be no security patches or updates for XP and most other software vendors will also drop support for it. Most new software probably won't even run on it.
I'd be very surprised if anyone with a PC running XP has hardware good enough to play modern games anyway - and if they do, they can easily upgrade to a newer operating system.
I think it makes absolutely no sense to try to support multiple different DirectX codebases when almost all modern PCs will run DirectX 11 quite happily and there's a working OpenGL option as a fallback anyway.
#60
To be clear, I'm all for dumping XP support if there is a clear advantage (for me) to do so. If the jump to D3D9Ex, or even D3D11, only results in minor performance gains then it's not a must-have for me.
That's not to say the work being done here isn't useful, because it certainly is. I just may not require it for any projects I'm working on at the moment. :]
04/05/2014 (12:16 pm)
I'm aware of XP's age and upcoming lack of support. I've long since made the jump to Win7 and will likely switch to Win8 soon. My concern is there's still a sizable portion of potential customers (and by customers I mean gamers with adequate hardware still running XP) that -haven't- made the switch and aren't likely to do so just because some indie game says they should. To be clear, I'm all for dumping XP support if there is a clear advantage (for me) to do so. If the jump to D3D9Ex, or even D3D11, only results in minor performance gains then it's not a must-have for me.
That's not to say the work being done here isn't useful, because it certainly is. I just may not require it for any projects I'm working on at the moment. :]
Lopuska
Torque3D has 3 cubemap types:
-Dynamic //OK
-Static(TextureFace) //OK
-Static(DDS) //Not Tested
I need to know how I can test dds cubemap loading from editor or code. Does anyone know how to help me to do that?
I have done a lot of changes in this part of code, and I need to test it just to verify the new functionalities!
P.S.
When I create a skybox, Torque always uses the second type of cubemap:
Static(TextureFace)
But I need to test the DDS loading.
Thanks for your help! :)