Cryptainer for TGE1.5/TGEA/TGB

Download Code File

-------------------- Change log --------------------
Initial release: v.1.0 - 2005-Mar-04
TGE1.5 Merge: v.1.1 - 2007-Mar-07

-------------------- Known BUGs --------------------
# Some unresolved memory issues while saving data into containers. See this thread for more info.

-------------------- Note --------------------
This resource is based on original "Cryptainer" resource (ID7276, not hosted anymore on GG) made by James Urquhart. I've adopted the resource to be compatible with TGE 1.5 SDK. I've not tested it with TGB/TGEA but it should be compatible, as the "core" is the same for all engines. But, if you want to merge it into TGB or TGEA you need to see the patch file and do manual merge, I don't have time to create patches for TGEA and TGB (if someone will do this job, I'll update this resource).

-------------------- Another Note --------------------
This resouce need more improvement - exposing functions to delete and copy files into script, hope someone from community will work on it. If you done ANY improvement, please post back here in comments your changes, and if they are good, I'll update the resource to include it.

-------------------- And Another Note --------------------
This resource I can rate as a 7 of 10 on "difficulty level" (quite hard one for beginners). So, if you are completely new in Torque or you are an artist, my recommendation is "think twice!" before you start playing with it. But, if you consider this resource as a method of learning Torque codebase - I can only wish you good luck with that. Looking carefully at the affected code you can understand on how the filestreams are working and many other aspects.

-------------------- Instructions --------------------
Instructions on how to install:

1) Download the zip

2) Patch the sources

Place the cryptainer.patch file (included in ZIP file) in your "torque" directory (where engine, example and lib folders are), and apply it, e.g. on linux :

patch -po < cryptainer.patch

Alternatively, if the patching fails, manually apply the patch.

In addition to patching the sources, you will need to make sure any custom code you have made which uses the "ResourceManager->openFileForWrite" function is altered to use the alternate version supplied with this resource, which returns a Stream*, and does not require a FileStream& to be passed to it. Also be sure that you use ResourceManager->closeStream() to close the stream. See the patch file of details on how this is done. The simplest and clear exaple is the changes in engine/game/banList.cc file.

In the sources I've made all changes commented in a way like:

//cryMod

if the changes affect only one line of code (old commented - new added OR just commented line OR just added new line).

//cryMod A
... some code
//cryMod Z

Where A and Z is start and end of changes. Again, looking at the sources you can easily read what was done.
For example, have a look at /engine/game/banList.cc:

void BanList::exportToFile(const char *name)
{
   //cryMod
   //FileStream banlist;
   Stream *banlist;

   char filename[1024];
   Con::expandScriptFilename(filename, sizeof(filename), name);
   //cryMod
   //if(ResourceManager->openFileForWrite(banlist, filename, FileStream::Write))
   if(banlist = ResourceManager->openFileForWrite(filename, FileStream::Write))
   {
      char buf[1024];
      Vector<BanInfo>::iterator i;
      for(i = list.begin(); i != list.end(); i++)
      {
         dSprintf(buf, sizeof(buf), "BanList::addAbsolute(%d, \"%s\", %d);\r\n", i->uniqueId, i->transportAddress, i->bannedUntil);
         //cryMod
         //banlist.write(dStrlen(buf), buf);
         banlist->write(dStrlen(buf), buf);
      }
   }

   //cryMod A
   //banlist.close();
   ResourceManager->closeStream(banlist);
   //cryMod Z
}

3) Incorporate libtomcrypt & bzip2
The patch includes modifications to the Makefile's so you can compile these library's in "straight out of the box".
However, in other environments, you will need to make appropriate project files for these static library's. Refer to the lib/targets.bzip2.mk and lib/targets.libtomcrypt.mk files for any extra compile options required.

You will need to download the sources of bzip2 (1.0.x), and libtomcrypt (0.92), from the following locations, and extract to the appropriate folders in lib/.

bzip2 downloads (this resource was compiled using bzip2 v1.0.4

LibTomCrypt downloads (this resource was compiled using LibTomCrypt 0.92, and I have included the libtomcrypt.zip file I've used [VC2003 .vcproj], add it to your Torque SDK solution and compile)

Due to the size limitation of files that can be uploaded, the libtomcrypt is not included in resource, and can be downloaded from separate download link.

4) Add the crypto files

NOTE: this step is already done for the Makefile's (presuming the patch applied successfully)

You will need to copy the following files into your engine/core/ folder, and add them to your project:
engine/core/resContainer.*
engine/core/resFilter.*
engine/core/filterState.*
engine/core/hash.h
engine/core/resourceFilters (whole directory)
lib/bzip2 (whole directory)
lib/libtomcrypt (whole directory)

NOTE: with the exception of E_tomcrypt.cc, P_none.cc, and C_zlib.cc, all the files in core/resourceFilters are optional.

Also remove the following files from your project:
core/zip* (*.cc/*.cpp & *.h)

If your using the Makefile's, copy them into your lib/ folder:
lib/targets.bzip2.mk
lib/targets.libtomcrypt.mk

Add the following to your include paths:
lib/libtomcrypt
lib/bzip2 (if including bzip2 compression)

And add the following static libs to your torque target:
libtomcrypt
bzip2

5) Compile
Provided you have incorporated the new files, aswell as the changes correctly, your copy of torque should compile cleanly with the new features.

6) Use it

Quick overview of the new console functions

setContainerKey(container, key) - sets the crypto key to use to decrypt encrypted assets from container.
setContainerHash(container, hash) - same as setContainerKey(), but uses a precomputed hash.
getHash(key) - returns computed hash for string "key".
setFilterFlags(flags) - sets the filter flags used for containers/files created from now onwards.
touchContainer(name) - creates an empty container file.
dumpCompressionHandlers() - prints a list of compression handlers compiled in.

Processing flags (1 and 1 only required)

$Container::PROCESS_BASIC
$Container::PROCESS_DELTA8
$Container::PROCESS_DELTA16
$Container::PROCESS_DELTA32
$Container::COMPRESS_ZLIB
$Container::COMPRESS_BZIP2
$Container::PROCESS_ALL (Value with all PROCESS_* and COMPRESS_* flags set)

Encryption flags (optional, others can be added by modifying enum)

$Container::ENCRYPT_BLOWFISH
$Container::ENCRYPT_TWOFISH
$Container::ENCRYPT_RIJNDAEL
$Container::ENCRYPT_XTEA
$Container::ENCRYPT_RC6
$Container::ENCRYPT_DES
$Container::ENCRYPT_ALL (Value with all ENCRYPT_* flags set)

To add files to a container, merely make sure the container exists, and write to it as if it was a directory:

// Set up container
setFilterFlags($Container::COMPRESS_BZIP2);
touchContainer("starter.fps/myContainer.aua");

// Enable encryption (optional)
setFilterFlags($Container::COMPRESS_BZIP2 | $Container::ENCRYPT_BLOWFISH);
setContainerKey("starter.fps/myContainer.aua","pies taste good");

// Setup file to write to and save something
new FileObject(testFile);
testFile.openForWrite("starter.fps/myContainer.aua/test.cs"));
testFile.writeLine("$var1 = \"It was from container!\";");
testFile.close();
testFile.delete();

// Testing the code
echo($var1); // will output nothing, eg empty line
exec("starter.fps/test.cs");
echo($var1); // will output "It was from container!"

NOTE: Containers can also contain directory's, or rather, paths. However, unlike files, directory entries can only be compressed.

To load files from a container, merely load them directly from the container; Containers collapse themselves to the root of the directory they are in, so you can also load the files that way (provided you restart torque, or use setModPaths() again). e.g:

// Try and execute the script in the container "starter.fps/myContainer.aua"
exec("starter.fps/myScript.cs");
// If exec() fails, make sure you set the correct key for the container, aka:
setContainerKey("starter.fps/myContainer.aua","pies taste good");
exec("starter.fps/myScript.cs");

As for deleting files, this is not exposed to script; The only time a container will delete a file is when it is replacing it with a new copy.

Have fun!