Asset Encryption/File Packing
by Bloodknight · in Torque 3D Beginner · 08/25/2010 (11:09 am) · 18 replies
Ive been searcing around both here and abroad for solutions and havent found anything of real use, the solutions presented on the GG site are incredibly old and almost guaranteed to not work with T3D, it would appear that the zip functionality that existed in TGEA was removed/disabled in T3D because of problems, and all the 3rd party software does little more than pile everything into one bag and then run some kind of 'virtual' file system in memory to access the files, the problem with this whole approach however is that if i add every single pack ive bought to a game i have something like 3gb of asssets not counting the collection of textures ive created myself, so a huge 3+ gigabyte single file is an unnaceptable solution to the problem.
In real terms i need the ability to pack up each folder in the art folder into a data file so that i have things like houses.dat players.dat and so on.
So my question really here is, what do you guys use? how do you use it and how does it effect performance.
I think i liked the idea of the zip recource what i can remember of it, and i'm approaching apoint whereby i'd like to send out early tech demos to people, but 'asset protection' is a stipulation in more than a few of my asset licenses.
another part of all this is the fact that i just dont want to release a game that has 20,000 files in it folder system, asset packing for the sake of tidyness is also a primary aim here
NB i dont want this thread to degrade into the usual "you cant protect your stuff *ever* because a 12 year old boy genius with 3 crays on his desktop can hack your assets in 30 seconds" i dont care, locks on house doors dont keep burglars out either, but they do fulfill a legal requirement and they do fulfill enough criteria to get insurance on your property, and that basic legal covering my ass is all i'm looking for here.
In real terms i need the ability to pack up each folder in the art folder into a data file so that i have things like houses.dat players.dat and so on.
So my question really here is, what do you guys use? how do you use it and how does it effect performance.
I think i liked the idea of the zip recource what i can remember of it, and i'm approaching apoint whereby i'd like to send out early tech demos to people, but 'asset protection' is a stipulation in more than a few of my asset licenses.
another part of all this is the fact that i just dont want to release a game that has 20,000 files in it folder system, asset packing for the sake of tidyness is also a primary aim here
NB i dont want this thread to degrade into the usual "you cant protect your stuff *ever* because a 12 year old boy genius with 3 crays on his desktop can hack your assets in 30 seconds" i dont care, locks on house doors dont keep burglars out either, but they do fulfill a legal requirement and they do fulfill enough criteria to get insurance on your property, and that basic legal covering my ass is all i'm looking for here.
About the author
#2
08/25/2010 (2:24 pm)
and just as i try to buy it to test it, the payment gateway dies. Its going to be one of those days... 
#3
We need to package up our assets for alpha testing.
Let me know what solution you find Bloodknight
08/25/2010 (4:18 pm)
I have that package and would like to know how your implementation goes!We need to package up our assets for alpha testing.
Let me know what solution you find Bloodknight
#4
10/06/2010 (10:40 am)
tried this again right now and it wont let me purchase via my usual method of paypal, si i guess i need to wait for beta 12 or version 2 or whenever they decide to fix all the working TGEA features they broke in T3D 
#6
10/06/2010 (8:30 pm)
It works, but its not without flaws. We cannot use it as it stands right now as some things fail to load. I can't go into further detail but it looks kick ass if we get these issues sorted.
#7
10/20/2010 (2:00 pm)
i'm trying to understand how to get molebox to offer the same functionality as the old zip resource did, but i'm having a 'gibbering idiot' week this week and i faile to understand even the simplest things :) 
#8
10/21/2010 (10:31 am)
genEpak doesnt work with Torque 3D 1.1b2, b3. I did use Phoenix Sentry for a while, but then they vanished so no longer supported if things go wrong.
#9
sadly it does have some issues, not showstoppers, but not ideal either. All data packages must be stored in the same folder as the executable, which means i have to pack the root folder with all the data files rather than leave them inside sub folders, but at least it works, and it more than satisfies any of the asset protection requirements of any model/image purchases.
ive not had anything failed to load as of yet, is it possible to expand on your issue, i did have an issue with config.cs but once i removed it from the data package that fixed itself.
11/24/2010 (9:05 pm)
out of necessity and lack of any other real and usable options i went and bought molebox.sadly it does have some issues, not showstoppers, but not ideal either. All data packages must be stored in the same folder as the executable, which means i have to pack the root folder with all the data files rather than leave them inside sub folders, but at least it works, and it more than satisfies any of the asset protection requirements of any model/image purchases.
ive not had anything failed to load as of yet, is it possible to expand on your issue, i did have an issue with config.cs but once i removed it from the data package that fixed itself.
#11
11/25/2010 (2:08 am)
every time i try to use paypal to buy genepak it fails, and i refuse to give my credit card details to an unknown entity
#12
Ive just gotten genepak to try, but the documentation indicates that it has the same feature set (files must be stored in root), 2 other factors that i dislike about this product before I even start to use it, one is the fact that it seem you have to hard code every packed file in the source, another is the fact that there's an untrusted(imo because i dont have the source) 3rd party application that has access to the passwords in the form of the genepak executable.
while both these products do *just* enough to satisfy the requirements of the most guarded art asset EULAs, they don't do so in quite as nice of a way as the old zip manager system did or am I the only one that doesn't want to see 10gb worth of packed files inside the root folder?
01/12/2011 (12:56 pm)
Well i'm officially pissed off at molebox as of today my update licence expires and the feature he said he was adding doesnt work. So thats $120 wasted in some respects, while the product works, it doesnt work the way i wanted it to which is having the ability to store the packed files in folders other than the root folder.Ive just gotten genepak to try, but the documentation indicates that it has the same feature set (files must be stored in root), 2 other factors that i dislike about this product before I even start to use it, one is the fact that it seem you have to hard code every packed file in the source, another is the fact that there's an untrusted(imo because i dont have the source) 3rd party application that has access to the passwords in the form of the genepak executable.
while both these products do *just* enough to satisfy the requirements of the most guarded art asset EULAs, they don't do so in quite as nice of a way as the old zip manager system did or am I the only one that doesn't want to see 10gb worth of packed files inside the root folder?
#13
Notice: By default VirtualMountSystem is disable due to the bug in engine.
Here is a thread how to enable/fix it: zipped folder access
01/12/2011 (2:53 pm)
We use password-protected ZIPs (build-in feature of Torque). In order to get it more secure, we generate passwords on-the-fly for every single archive, so there are no fixed password.Notice: By default VirtualMountSystem is disable due to the bug in engine.
Here is a thread how to enable/fix it: zipped folder access
#14
Zip folder access, fails and silently so, this may well be because I'm trying to use zip folder zips in the editor, or simply because I missed something, either way, I'm one of those people that wishes software would send fireworks when it fails rather than shrugs its shoulders and slinks of into the void...
I did get genEpak working, but only partially, if you pack everything it dies using some kind of runtime error, I suspect that its trying to write files and failing in spectacular fashion. I made a pack of the tools folder which works fine as expected, but if i try to do anything clever, like... pack individual art/shapes/* folders it does the whole silent fail on me again.
Molebox, despite my earlier grumbles, and I have a few more, is the 'best fit' for what I want at the moment, I can pack the art/shapes/* folders separately even if I do have to store them in the root folder (guy said he would fix that then promptly ignored all my attempts at communication) and I refuse to pay full price again for a single feature which in my opinion should have been part of the feature set for $120.
I was going to try the smart packer thing, but I'm not sure I like the company's ethics or the implied associated attitude if I'm honest.
Well I hope this at least is informative to people searching for this kind of stuff, and if anybody has any other suggestions (no, I'm not going to write my own so lets not take that road) almost always willing to try something new (software... I mean new software.. freak.. you know who you are!)
01/17/2011 (9:59 pm)
This is turning into some kind of blog I keep updating...Zip folder access, fails and silently so, this may well be because I'm trying to use zip folder zips in the editor, or simply because I missed something, either way, I'm one of those people that wishes software would send fireworks when it fails rather than shrugs its shoulders and slinks of into the void...
I did get genEpak working, but only partially, if you pack everything it dies using some kind of runtime error, I suspect that its trying to write files and failing in spectacular fashion. I made a pack of the tools folder which works fine as expected, but if i try to do anything clever, like... pack individual art/shapes/* folders it does the whole silent fail on me again.
Molebox, despite my earlier grumbles, and I have a few more, is the 'best fit' for what I want at the moment, I can pack the art/shapes/* folders separately even if I do have to store them in the root folder (guy said he would fix that then promptly ignored all my attempts at communication) and I refuse to pay full price again for a single feature which in my opinion should have been part of the feature set for $120.
I was going to try the smart packer thing, but I'm not sure I like the company's ethics or the implied associated attitude if I'm honest.
Well I hope this at least is informative to people searching for this kind of stuff, and if anybody has any other suggestions (no, I'm not going to write my own so lets not take that road) almost always willing to try something new (software... I mean new software.. freak.. you know who you are!)
#15
01/22/2011 (2:10 am)
We're currently trying out this product www.smartpacker.nl its a bit like molebox but has other functionality and seems very well supported. Another member on GG also tried out the trial without any problems other than the possibility of increased load times but that's to be expected from any solution like this. 
#16
I checked with the developers of smart packer re @S2P
and his response to an upcoming update in a few months (as of today):
I like the sounds of this one and removes the need for zip
and combines with an update facility. I'm waiting for this update now.
10/06/2011 (10:45 am)
Just an update to this..I checked with the developers of smart packer re @S2P
and his response to an upcoming update in a few months (as of today):
Quote:
The first version will allow patching files (patch files probably have a
file size 30%-50% compared to original file). In later versions this will be
further optimized.
About your other question.
Packed file groups (which are encrypted and compressed archives) are not
required to be in the root dir. It will be very flexible.
Kind regards,
Michael
Smart Packer Solutions
www.smartpacker.nl
I like the sounds of this one and removes the need for zip
and combines with an update facility. I'm waiting for this update now.
#18
The new features are:
1. New run-time engine
2. On demand feature (will extract file on demand)
3. Version info can be altered
4. Unicode support in GUI and run-time engine
5. Link to web page for free and commercial plug-ins
Improvements:
1. Adding included files easier
2. Overall better performance
3. GUI improvements (we hope :)), more to come in future!
3. Minor bug fixes
Of course it is a free upgrade for registered customers!
Smart Packer Pro is now also available as MicroISV and Company license. See our store for more details.
The new product mentioned in the post of John is currently in development.
If you have questions and/or remarks just let me know.
01/23/2012 (2:01 am)
FYI! Smart Packer Pro 1.7 is now available!The new features are:
1. New run-time engine
2. On demand feature (will extract file on demand)
3. Version info can be altered
4. Unicode support in GUI and run-time engine
5. Link to web page for free and commercial plug-ins
Improvements:
1. Adding included files easier
2. Overall better performance
3. GUI improvements (we hope :)), more to come in future!
3. Minor bug fixes
Of course it is a free upgrade for registered customers!
Smart Packer Pro is now also available as MicroISV and Company license. See our store for more details.
The new product mentioned in the post of John is currently in development.
If you have questions and/or remarks just let me know.
Associate Steve Acaster
[YorkshireRifles.com]
Also 12 year old haxx0rz :P