Torque3D's Import System

#1 raa,

You won't see it in an MIT licensed T3D, Assimp is released under BSD so the license thing could be a problem.

Ron

#2 Um, the BSD license is very permissive. It's definitely compatible with MIT.

#3 Probably the first place to start, if one were so inclined, technically knowledgeable, and had the time (Which means 2 strikes against me there, unfortunately.) would be the inverse of www.garagegames.com/community/blogs/view/22526 where instead of killing off the engine portions not related to .dts and conversions to it, you'd extract it into a sub-module (likely with wrappers to reference a filetype lookup. Not unlike the engine does with texture file formats.).

#4 @Azaezel, I seriously think DTS needs to go. DTS is actually holding back complex models and is a unsupported 3D format overall. Maybe in Torque3D 4.0, the import system will be overhauled to be compatible with Assimp's import methods. The import system needs to be overhauled.

@Ron, as Daniel said, the BSD liscense is very compatible with MIT, or at least for Torque's purpose.

#5 They're mutually compatible. They're both basically 'do what you like' licenses.

I agree, we should put an expiry date on DTS. I mean, it's one of those things that should be kept around as a plugin since people have existing assets in DTS.

#6 DTS is never going away as it is the engine's internal format of all 3D models hence why cached imported models are <originalName>.cached.dts files. All game engines that are worth anything have their own internal model formats for performance reasons as the format is designed that best suites how the engine uses and renders models.

Provided that Assimp does have a very good neutral model format API then it would be great to have in Torque 3D to finally not require other model formats to go through programs like Blender or 3DS Max for modifications before importation as .DAE files into T3D. Cause quite frankly .DAE has been one terrible mess compared to .DTS file history.

#7 Right, the internal DTS representation probably needs to stay. Nathan, how familiar are you with the DTS file format and its use in T3D?

#8 Okay, maybe it should stay, but another import module using Assimp's should be implemented.

#9 All the code used to construct a DTS from a DAE is located in ts/loader - pay special attention to the TSShapeLoader class.

There is also the TSShapeConstruct class which is essentially the backend for the shape editor.

From a technical perspective, I don't see anything wrong with DTS itself apart from it could do with a tidyup in a few places (like the loading code, eek!). Also it wouldn't hurt to implement support for some more modern features like IK Rigging and HW Skinning.

Technicalities aside, the root problem of course with implementing something like this is finding the time and resources to do it, which might be better spent just fixing the existing code to do what you need it to do.

#10

Quote:just fixing the existing code to do what you need it to do

Unfortunately I think that policy is what has gotten the engine source into the state it's in :P. Slight exaggeration, but if we're talking about the future of T3D, rather than features raa needs for raa's game, then I think we need to find the time and resources.

#11 Well, I do think it needs to be added for the future of T3D. I am currently not working on a game right now. So I'm not looking for extra features from everyone, I am trying to add features to T3D so people can do amazing things. That is currently why I'm trying to port GMK but with less bugs to T3D.

Look at this video, can we try to push T3D to this point?


All I'm saying is this is something for T3D 4.0 developers to look at. Even though we all technically are, the main developers (one main dev being you) should look at this. Its time and resources, but isn't that what is needed for everything?

#12 Am I the only one that's had zero issues with DAE ? Maybe I've just got lucky this whole time but I have art assets coming from both blender and 3ds max into DAE with no issues.

Regardless, more formats = more better. I took a look at Assimp, and it really is gift wrapped. I mean, look at the python example:

scene = pyassimp.load(MODEL)
print "  meshes:", len(scene.meshes)
print "  materials:", len(scene.materials)
print "  textures:", len(scene.textures)

It really shouldn't be too difficult to feed assimp's data into tsShapeLoader. If I was more familiar with tsShapeLoader or in need of additional formats, I'd hop on it, but I can't take this on at the moment. Too much on my plate. If no one tackles it in the coming months I'll take a shot at it when I get some time.

p.s. @ raa : If you took unreal engine 4 and replaced all those models in the video with collada models, it would still look the same. The video you posted is irrelevant to the discussion at hand.

#13 @Andrew:
Afaik there is one problem with dae and this is the visibility animation. The last time I tried it did not import it to the dts model. I don't know if this is caused by the dae exporter, but for models that use visibility I still export to dts instead of dae.

#14 Assimp, yeah I mentioned this over a year ago. It's nice to see that others are recognizing the benefit of it.
I wouldn't replace .DTS or .dae tho, like Andrew I haven't had any problem with models that are properly made, maybe it's just me but I'm happy with DTS and dae..I do see the benefit (mainly for others) for assimp..
Now getting the same visual quality as unreal 4, a few things need to happen.
First unreal 4 uses Physically Based Rendering, so upgrading to PBR for one.
Second, the DX 11 and openGL 4+, would need to be in place. I was told by a few different programmers that it is highly important to have DX10-11 and/or openGL4+ layer complete because anything lower can't run it (PBR) with good framerates, or the same what you see in modern gen games...
Well that's my 2 cent, I'm sure the SC has a good vision and has a good understanding of the direction the engine should go into to be competitive :)

#15 To be honest, I don't think Assimp would be hard either. If I were experienced enough with C++ and Torque's API, I would be right on it to. At the moment, time isn't all that of a problem to me, just lack of experience. Assimp would not be a hard implementation, I know enough to say that. If anyone knows the shape base of T3D, it wouldn't hurt to start or try to implement Assimp.

As for UE4, DX11 and OpenGL are imminent. Although Luis isn't working on OpenGL 4, but rather 2.1, it won't be a hard upgrade. At all. Physically Based Rendering will take time, but if that is finished, I don't know what else would be needed to race with the other stunning engines. The only thing I can think of is one big amazing community graphical demo that would push the limits of T3D.

#16 I decided to take a shot at it today and see how far I could get. Not bad progress but I'm getting caught up in linker errors that I don't feel like debugging at the moment so I've committed everything to this branch:

github.com/andr3wmac/Torque3D/tree/assimp

So, first off, assimp loves boost. I hate boost. I hope boost dies in a fire. Luckily they included mini versions of boost for the specific things they need. The ASSIMP_ENABLE_BOOST_WORKAROUND doesn't really work. There's multiple spots in the code where it just straight up includes boost and these won't work. So, I altered those spots to use their workaround and after a little back and forth I got the library to compile. I tested it with their openGL sample and its working fine.

Next, I moved it into torque project generator and filled in those blanks. It generates the project file as expected and the library will compile. This is where things started to get bumpy. The build won't compile because of "defaultlib 'MSVCRT' conflicts with use of other libs;" so I excluded that with /NODEFAULTLIB:MSVCRT in command line options for the project and then it compiled fine.

I started working on the actual loading code a bit, setting up a assimpShapeLoader class and TORQUE_ASSIMP flag. I added an entry point into tsShape.cpp so if the extension is not .dae or .dts before failing to load it will send it to assimp to see if it can be loaded from there.

This is where I hit my next problem. Including the headers is fine, but as soon as I make use of the aiImportFile() function from assimp, I get a massive list of linker warnings/errors.

I've had enough for today. It's all on github.

#17 Boost? No. Just no. Anything that invokes boost is a god awful terrible idea.

#18 Perhaps I was unclear in my previous post. They included a boost workaround that avoids a dependency on boost. It just didn't work right out of the box, and I fixed it. So, no boost.

The downside to this is the library becomes single threaded, but that's a fair trade off for no boost.

#19 Good work Andrew! Please keep us updated. Thanks.

#20 I put in a couple hours on this today. I gave up trying to get assimp to build as part of torque. I figure it's best to spend time on that later when it's actually proven itself worthy.

I installed the SDK and pointed a torque project to its includes/libs and dropped Assimp32.dll in the game folder. Loaded up with no issues. I have an object in the map that points to a .blend file with 4 cubes in it. As of right now all it does is spit out the information on the model in the console but it's off to a good start. I built an assimpShapeLoader class around it and I've been filling in the blanks.

It's nice because assimp provdes a root node to follow the hierarchy in the model which is exactly how collada + tsShapeLoader function so it doesn't look like it'll be too bad. It seems like every time I've hit a snag where I think "how am I going to generate that information" it turns out assimp already provides it. It's a very nice library so far.

That's enough for today though, I need to spend time on other things. I'm just going to spend a little time on this each day and it will slowly progress towards functional.