DTS cache and DSQ files
by James Brad Barnette · in Collada Test · 01/06/2009 (12:25 pm) · 22 replies
I have noticed that when you load a Collada it makes a DTS cache file. I assume that you could then take that and use that once it is created though I have not tried it yet. Has anyone tried this with a animated mesh yet? does it right out cache files for the DSQ as well?
About the author
#2
01/06/2009 (1:01 pm)
@Chris - So the cached DTS includes all the DSQs. Does this include DSQs loaded as part of the TSShapeConstructor? Or is it just animations that were embedded in the original DAE? 
#3
01/06/2009 (1:16 pm)
No - only animations embedded in the original DAE. The idea is that loading the cached DTS gives the exact same result as loading the DAE. This currently fails when it comes to materials however, since the DAE file can store much more information than the cached DTS.
#4
What about stand alone DSQs? Can we do that with Collada?
The Material settings within the DTS format are pretty dead... no one uses them for anything other than getting the material name. Those are hold overs from TGE which had no Material system.
01/06/2009 (1:20 pm)
Yea... that's the best logical solution.What about stand alone DSQs? Can we do that with Collada?
The Material settings within the DTS format are pretty dead... no one uses them for anything other than getting the material name. Those are hold overs from TGE which had no Material system.
#5
You mean have a collada file that only contains animation data (no geometry or materials)? Sure! The new TSShapeConstructor enhancements allow you to load an animation from a DAE file and add it to the base shape.
01/06/2009 (3:03 pm)
Quote:What about stand alone DSQs? Can we do that with Collada?
You mean have a collada file that only contains animation data (no geometry or materials)? Sure! The new TSShapeConstructor enhancements allow you to load an animation from a DAE file and add it to the base shape.
#6
Is there a reason that the system be it engine or standalone could not save the animation could not be made to convert Collada animation_clips and convert each one into a .dsq or put them in to another .dts file. DTS?
Would be preferred to be honest as it maintains the existing asset system.
01/06/2009 (6:56 pm)
Hmm does it have to be a .dae file though? In my as far as I'm concerned it is not acceptable to ship a .dae file. Is there a reason that the system be it engine or standalone could not save the animation could not be made to convert Collada animation_clips and convert each one into a .dsq or put them in to another .dts file. DTS?
Would be preferred to be honest as it maintains the existing asset system.
#7
The Collada loaded caches what it loads as DTS (and presumably the DSQ) files.... you only need to ship those.
01/06/2009 (7:15 pm)
@James - We've found that DAE files have been much faster to load than we expected... but regardless... you do not have to ship DAE files.The Collada loaded caches what it loads as DTS (and presumably the DSQ) files.... you only need to ship those.
#8
Does this mean that the DTS/DSQ format is changing, or has changed since DTSPlus! SDK was released with the new TSShapeConstructor?
The way I understand it[Resource Manager]; it seems that when a TSShapeConstructor 'builds' a shape[loads DSQ files into the DTS], that the result is a DTS shape with animation, the DSQ files and the DTS 'rawData'[DTS and DSQ, or DTS and DAE files, or DAE and DAE files, or DAE and DSQ files] are compiled and stored in Resource Manager....and if you've compiled the TSShapeConstructor at Runtime....you don't need any of the 'rawData' anymore?? And thus, the 'editable' DAE files[shape OR animation]......would not need to be shipped....this is how I understand it. Now, what happens to the data in the Resource Manager....once compiled and the server destroyed?!?
This is something I've often wondered about with the Resource Manager and any 'rawData' files[DTS/DSQ]....what really happens to them at runtime and after. Once compiled into a DTS shape with motion data[DSQ]....the DTS shape is never referenced again??
01/07/2009 (5:44 am)
Quote:The cached DTS file can be re-used, although because the DTS file format has changed it cannot be loaded in STP yet.
Does this mean that the DTS/DSQ format is changing, or has changed since DTSPlus! SDK was released with the new TSShapeConstructor?
The way I understand it[Resource Manager]; it seems that when a TSShapeConstructor 'builds' a shape[loads DSQ files into the DTS], that the result is a DTS shape with animation, the DSQ files and the DTS 'rawData'[DTS and DSQ, or DTS and DAE files, or DAE and DAE files, or DAE and DSQ files] are compiled and stored in Resource Manager....and if you've compiled the TSShapeConstructor at Runtime....you don't need any of the 'rawData' anymore?? And thus, the 'editable' DAE files[shape OR animation]......would not need to be shipped....this is how I understand it. Now, what happens to the data in the Resource Manager....once compiled and the server destroyed?!?
This is something I've often wondered about with the Resource Manager and any 'rawData' files[DTS/DSQ]....what really happens to them at runtime and after. Once compiled into a DTS shape with motion data[DSQ]....the DTS shape is never referenced again??
#9
01/07/2009 (5:53 am)
What is STP "Standard Torque ***"?
#10
From my understand the are saying the version of it that is in our test build of T3D is different than what is in standard TGEA 1.8
01/07/2009 (6:03 am)
Quote:
Does this mean that the DTS/DSQ format is changing, or has changed since DTSPlus! SDK was released with the new TSShapeConstructor?
From my understand the are saying the version of it that is in our test build of T3D is different than what is in standard TGEA 1.8
#11
This would perhaps provide a 3rd Party type app for a 'wider' grphix market, outside the Torque Community, Derek....;), eh? STP needs some uberLove, soon!
01/07/2009 (6:17 am)
STP= ShowToolPro, as I know it...and it would be great for it to read Collada as well! And have some of those cool...methods for calling. addNode...etc...can we have them visualized in T3D like the DNA resource from the Community. That feature is in our codeBase, and I tell ya, makes viewing shapes with rigs much easier in the engine....like is a Eye node inside the bounding box or not?! Instant visual feedback inside the engine is paramount when developing, inho.This would perhaps provide a 3rd Party type app for a 'wider' grphix market, outside the Torque Community, Derek....;), eh? STP needs some uberLove, soon!
#12
Confused:
Can someone confirm this? What is the solution for separated models and animation? It sound to me like we are back at having to ship with either
a: a bloated DTS full of animation.
or
b: a DTS file for a model and a .dae file for our animation that anyone can take and use anyway they want.
is it that it is just too difficult get animation out of the .dae file? If you can put it in a dts file then I would think as long as the sequences are defined it should not be much more difficult to write them out to dsq files than it was to pull the mesh in and put it out as a DTS.
can we get a definative answer on this.
If the answer is B, again I must STRONGLY advise against this. People will steal, hack, and manipulate a dae file in a heartbeat.
01/07/2009 (6:21 am)
Quote:
Tom:
The Collada loaded caches what it loads as DTS (and presumably the DSQ) files.... you only need to ship those.
Quote:
Chris:
The collada loader does not write DSQ files, but an animated mesh will cache a DTS file containing the animation data.
Confused:
Can someone confirm this? What is the solution for separated models and animation? It sound to me like we are back at having to ship with either
a: a bloated DTS full of animation.
or
b: a DTS file for a model and a .dae file for our animation that anyone can take and use anyway they want.
is it that it is just too difficult get animation out of the .dae file? If you can put it in a dts file then I would think as long as the sequences are defined it should not be much more difficult to write them out to dsq files than it was to pull the mesh in and put it out as a DTS.
can we get a definative answer on this.
If the answer is B, again I must STRONGLY advise against this. People will steal, hack, and manipulate a dae file in a heartbeat.
#13
01/07/2009 (6:29 am)
Well...perhaps so, at the moment you can do the same with DTS/DSQ[in a heartbeat]....those formats are not encrypted at all, are they? I can read and convert both pretty easily at the moment.....you want the SDK dsq's....? Done. You want the SDK dts....? Done.
#14
what I'm saying is that with a dae you have animation that is out in the open. all some has to do is omport it into MAya or XSI or something like that and then apply it to their characters. then they can use it in whatever they want.
01/07/2009 (7:26 am)
I don't know of any modeling application that have dts or dsq import capability. Plus the are not human readable like dae is. dae is basicly xmlwhat I'm saying is that with a dae you have animation that is out in the open. all some has to do is omport it into MAya or XSI or something like that and then apply it to their characters. then they can use it in whatever they want.
#15
I've been houndDogging these formats intensely the past 3 years....as I said, we got's more than 31 flavors here!
Sure, someone can always plagiarize work...don't see many that get away with it...
...I also have a 'tool' that reads back more DSQ data than the STP does...how shocking!, lol...
01/07/2009 (9:18 am)
Here's 2:: U3D Imports/Exports DTS/DSQ, Shaper Imports DTS/DSQ.I've been houndDogging these formats intensely the past 3 years....as I said, we got's more than 31 flavors here!
Sure, someone can always plagiarize work...don't see many that get away with it...
...I also have a 'tool' that reads back more DSQ data than the STP does...how shocking!, lol...
#16
Whenever the collada loader loads a DAE file, it will generate a DTS file containing the exact same data. As far as the collada loader is concerned, the cached DTS is merely a faster way to load the DAE.
If you want to have your animations in separate files, export them to separate DAE files, and they will be cached to separate DTS files (no geometry, so almost the same as a DSQ). TSShapeConstructor has been updated such that animations can be pulled from DTS or DAE files in the same way as DSQs.
Having said that, I can definitely see a real benefit to a standalone converter app that could become part of the 3d app export process.
01/07/2009 (12:34 pm)
There is no need to ever ship a DAE file.Whenever the collada loader loads a DAE file, it will generate a DTS file containing the exact same data. As far as the collada loader is concerned, the cached DTS is merely a faster way to load the DAE.
If you want to have your animations in separate files, export them to separate DAE files, and they will be cached to separate DTS files (no geometry, so almost the same as a DSQ). TSShapeConstructor has been updated such that animations can be pulled from DTS or DAE files in the same way as DSQs.
Having said that, I can definitely see a real benefit to a standalone converter app that could become part of the 3d app export process.
#17
Thanks for clearing that up. So DTS can perform the function that DSQ used to. That is perfect!
[quote]
Having said that, I can definitely see a real benefit to a standalone converter app that could become part of the 3d app export process.
[quote]
yeah, I have some around to seeing the logic of both ways but I have to say that it would be nice to have it. or at least some super lightweight version of the demo that has almost no function except it shows you the model and has command line stuff exposed to that it can be scripted from within the art pipeline.
I was thinking about this and perhaps the old Toque show tool could be ported to T3D. not the pro but the one that used to come with TGE. that was very basic. this way you could fire it off from you 3d app and then it would pop up and show your model in engine in a little window and then you hit exit when you are done. then you could go to the directory and there would be all of you file that it has created.
Just a thought though.
As long as I can get my stuff from XSI in to toque I'm happy.
@Derek or someone from GG:
Just out of curiosity have you guys eve considered buying STP from DAve and then porting it to T3D and adding material editing and collada import.
Just a thought. could kill many birds with one stone.
01/07/2009 (1:20 pm)
@ChrisThanks for clearing that up. So DTS can perform the function that DSQ used to. That is perfect!
[quote]
Having said that, I can definitely see a real benefit to a standalone converter app that could become part of the 3d app export process.
[quote]
yeah, I have some around to seeing the logic of both ways but I have to say that it would be nice to have it. or at least some super lightweight version of the demo that has almost no function except it shows you the model and has command line stuff exposed to that it can be scripted from within the art pipeline.
I was thinking about this and perhaps the old Toque show tool could be ported to T3D. not the pro but the one that used to come with TGE. that was very basic. this way you could fire it off from you 3d app and then it would pop up and show your model in engine in a little window and then you hit exit when you are done. then you could go to the directory and there would be all of you file that it has created.
Just a thought though.
As long as I can get my stuff from XSI in to toque I'm happy.
@Derek or someone from GG:
Just out of curiosity have you guys eve considered buying STP from DAve and then porting it to T3D and adding material editing and collada import.
Just a thought. could kill many birds with one stone.
#18
If so i would suggest we still keep the DSQ naming convention for animation files without geometry... just so its easy to tell those are just animations.
01/07/2009 (3:30 pm)
@Chris - Interesting... so have DSQ files just been DTS files without geometry all this time? I didn't know that one. If so i would suggest we still keep the DSQ naming convention for animation files without geometry... just so its easy to tell those are just animations.
#19
No - the file formats are slightly different, but a DTS file does not need to contain geometry, so you can use it as a container for animation only data.
I agree with you though => animation only DAEs should be cached to a DSQ rather than a DTS for consistency.
01/07/2009 (3:48 pm)
Quote:@Chris - Interesting... so have DSQ files just been DTS files without geometry all this time? I didn't know that one.
No - the file formats are slightly different, but a DTS file does not need to contain geometry, so you can use it as a container for animation only data.
I agree with you though => animation only DAEs should be cached to a DSQ rather than a DTS for consistency.
#20
01/07/2009 (8:35 pm)
@Chris - You know it is neat that you also now support loading a DTS as a sequence in the TSShapeConstructor. This seems to mean i can steal the embeded animations from one DTS and add it to another. Its not the best idea for a shipping game... but a neat trick if your in a pinch. :) 
Associate Chris Robertson
The collada loader does not write DSQ files, but an animated mesh will cache a DTS file containing the animation data. The cached file is used only to speed up loading collada files (since DTS will load much faster than collada).