Best way to package a game for internet e-distribution?
by Maria C R Harrington · in Torque 3D Professional · 02/20/2011 (4:06 pm) · 17 replies
I am looking for advice on the best practices to construct successful user experience in downloading and installing a T3D game, (read fast and without error).
Given, file size without compression is large: art, sound and scripts, (e.g. installation package 250 MG)
1) What are the best ways to optimize while building the game?
2) Once files are optimized in-game, what are the best ways to compress files for e-delivery?
3) Thoughts on DRM?
I am looking at Crispy Software's genEpak, but what others have been used?
Given, file size without compression is large: art, sound and scripts, (e.g. installation package 250 MG)
1) What are the best ways to optimize while building the game?
2) Once files are optimized in-game, what are the best ways to compress files for e-delivery?
3) Thoughts on DRM?
I am looking at Crispy Software's genEpak, but what others have been used?
About the author
Virtual Field Trips,LLC specializes in the design, production and publication of simulations and virtual environments for learning and entertainment. Simulations for Exploration, Discovery, and Intrinsic Learning Designed for the Young Child in All of Us.
#2
If you don't need alpha, you can reduce the amount of data in a PNG by 25% by not saving the alpha channel. Compressed size may be smaller. If you want to squeeze even more out, use pngout. Massive gains are possible, as some uniformly coloured images may actually not be as optimised as you think by your image manipulation program of choice. I've seen images with mainly one colour and a border drop from 300k to 50k. More commonly I save 20-50% on larger images, and 90% on tiny images (because something in libpng makes it give up easily, I guess - or maybe it's all the metadata).
A good installer will further compress files as one block, saving you another bit of filesize. InnoSetup and NSIS get two thumbs up from me. Don't bother with one-filing everything. You don't really gain anything doing that. The size and compactness of the distribution is far more important than what's installed on the user's system. Just make sure the uninstaller is good.
If you're using Steam, eSellerate, Impulse or FastSpring, they have various options for DRM. Depending on your target market, piracy may be extreme, making that day or more delay worth it. Generally somebody has to buy it before cracking it.
If your target market is the educational sector, you just need to find out where the pirates are, then revenge is a phone call away :P
(But they're of course more likely to buy bulk when they have some sense!)
02/20/2011 (6:25 pm)
250MB isn't much. I'll download ten game demos like that a day and forget about them :)If you don't need alpha, you can reduce the amount of data in a PNG by 25% by not saving the alpha channel. Compressed size may be smaller. If you want to squeeze even more out, use pngout. Massive gains are possible, as some uniformly coloured images may actually not be as optimised as you think by your image manipulation program of choice. I've seen images with mainly one colour and a border drop from 300k to 50k. More commonly I save 20-50% on larger images, and 90% on tiny images (because something in libpng makes it give up easily, I guess - or maybe it's all the metadata).
A good installer will further compress files as one block, saving you another bit of filesize. InnoSetup and NSIS get two thumbs up from me. Don't bother with one-filing everything. You don't really gain anything doing that. The size and compactness of the distribution is far more important than what's installed on the user's system. Just make sure the uninstaller is good.
If you're using Steam, eSellerate, Impulse or FastSpring, they have various options for DRM. Depending on your target market, piracy may be extreme, making that day or more delay worth it. Generally somebody has to buy it before cracking it.
If your target market is the educational sector, you just need to find out where the pirates are, then revenge is a phone call away :P
(But they're of course more likely to buy bulk when they have some sense!)
#3
Ive used genepak and it works fine within reason, i currently use molebox vs which also works fine within reason, theres a thread somewhere where i'm quite harsh about all solutions, you need to pick one which best suits your workflow/needs failing that, you can maybe customise the genepak and zip filesystem to your own specs (no source to genepaks packing utility is disapointing and i think limiting)
DRM is outright evil, DRM funds terrorists and drug lords, as well as child slave labour camps and human trafficing... just say no
02/21/2011 (12:52 am)
I use inno setup and a patch installer system for packing/installing the deliverables.Ive used genepak and it works fine within reason, i currently use molebox vs which also works fine within reason, theres a thread somewhere where i'm quite harsh about all solutions, you need to pick one which best suits your workflow/needs failing that, you can maybe customise the genepak and zip filesystem to your own specs (no source to genepaks packing utility is disapointing and i think limiting)
DRM is outright evil, DRM funds terrorists and drug lords, as well as child slave labour camps and human trafficing... just say no
#4
@ Bloodknight - Molebox looks very promising - Thanks!
If genPack can give a 50% reduction in file size, than that is significant - my demo download without DRM was 15 min, with the DRM was 30 min. Given that 70% of downloads are abandoned by end users due to "long" wait times, I need to do everything possible to reduce that time to less than 5 min. If I get that 50% reduction with genPack, I still need 30% more. Any more ideas?
How to implement streaming?
Dare I get into this topic on DRM? (my husband and I debate this one all of the time!). I think that DRM is only evil if it is used for evil... meaning that in the hands of the indie, it can protect the indie from those forces you mentioned... see the work over at Creative Commons.... It can still be FREE, just protected... like what I plan to offer to Teachers.
On the digital distribution channels... once one signs over rights to distribute with a third party, what do you give up? I never, ever want to sign over exclusive rights... then you need to integrate with their e-distribution and DRM system? What in the world does that involve. The only advantages I can see are the marketing, their brand, and maybe faster downloads?
02/21/2011 (7:05 am)
@ Steve, thanks! I am using the Inno installer (and fixed the T3D - I think), so both give compression of 60%, but that is not enough.@ Bloodknight - Molebox looks very promising - Thanks!
If genPack can give a 50% reduction in file size, than that is significant - my demo download without DRM was 15 min, with the DRM was 30 min. Given that 70% of downloads are abandoned by end users due to "long" wait times, I need to do everything possible to reduce that time to less than 5 min. If I get that 50% reduction with genPack, I still need 30% more. Any more ideas?
How to implement streaming?
Dare I get into this topic on DRM? (my husband and I debate this one all of the time!). I think that DRM is only evil if it is used for evil... meaning that in the hands of the indie, it can protect the indie from those forces you mentioned... see the work over at Creative Commons.... It can still be FREE, just protected... like what I plan to offer to Teachers.
On the digital distribution channels... once one signs over rights to distribute with a third party, what do you give up? I never, ever want to sign over exclusive rights... then you need to integrate with their e-distribution and DRM system? What in the world does that involve. The only advantages I can see are the marketing, their brand, and maybe faster downloads?
#5
You can make it so there is less to compress, like I mentioned above with optimisation of graphics. You can also try reducing polygons on models as far as you can bear and see if it gives any significant reductions in size.
Distribution channels differ, but generally ones like Steam and Impulse don't usually make you sign exclusivity deals or sign over anything. eSellerate and FastSpring are mere shop frontends, and leave you with the full rights. They handle payment for about 5-6% of your product price. But all of these are generally for paid products.
If you're giving something away, you've got Intel AppUp for Windows, and the Mac App Store. There's no exclusivity deal on the MAS either, just some rules to follow for the version distributed through them. Free or paid does not matter for either store.
DRM: There are few valid reasons for using it in the first place. These are the ones, in my opinion (and my word is law, y'know :):
1.Exclusive demos you want only a select few individuals to have, when a reduced build isn't possible.
2.Delaying the pirate distribution of a paid product.
The MAS comes with a few measures you need to implement to fulfil #2. FastSpring has built-in support for several types of serial key protection - you implement the code, upload a master key and it generates per-user keys. I've used this for AquaticPrime, and that's a fairly decent system (but Mac only - a few commercial solutions exist for Windows).
The only cases I know for certain involve signing over products would be when going though certain publishers who then arrange distribution on XBLA, MAS or large portals for you. They might give you an SDK to implement, or they may simply bake your binaries into some custom DRM.
I'm not sure how Big Fish and PopCap do it, but they also have their own DRM. I don't think they have any exclusivity requirements. PopCap are generally only producing in-house stuff, though. Big Fish is more of a classic portal. I know several people on these forums have experience with at least some of the aforementioned distribution portals.
02/21/2011 (8:04 am)
You can't compress better than the best compression :)You can make it so there is less to compress, like I mentioned above with optimisation of graphics. You can also try reducing polygons on models as far as you can bear and see if it gives any significant reductions in size.
Distribution channels differ, but generally ones like Steam and Impulse don't usually make you sign exclusivity deals or sign over anything. eSellerate and FastSpring are mere shop frontends, and leave you with the full rights. They handle payment for about 5-6% of your product price. But all of these are generally for paid products.
If you're giving something away, you've got Intel AppUp for Windows, and the Mac App Store. There's no exclusivity deal on the MAS either, just some rules to follow for the version distributed through them. Free or paid does not matter for either store.
DRM: There are few valid reasons for using it in the first place. These are the ones, in my opinion (and my word is law, y'know :):
1.Exclusive demos you want only a select few individuals to have, when a reduced build isn't possible.
2.Delaying the pirate distribution of a paid product.
The MAS comes with a few measures you need to implement to fulfil #2. FastSpring has built-in support for several types of serial key protection - you implement the code, upload a master key and it generates per-user keys. I've used this for AquaticPrime, and that's a fairly decent system (but Mac only - a few commercial solutions exist for Windows).
The only cases I know for certain involve signing over products would be when going though certain publishers who then arrange distribution on XBLA, MAS or large portals for you. They might give you an SDK to implement, or they may simply bake your binaries into some custom DRM.
I'm not sure how Big Fish and PopCap do it, but they also have their own DRM. I don't think they have any exclusivity requirements. PopCap are generally only producing in-house stuff, though. Big Fish is more of a classic portal. I know several people on these forums have experience with at least some of the aforementioned distribution portals.
#6
02/21/2011 (8:17 am)
thanks everyone!... also, is there a way to compile all of the game assets into one DLL?
#7
I think adding DRM as an anti piracy measure is a pointless exercise for the indie studio, unless you expect to make like 75% of your sales in the first 2-4 weeks.
02/21/2011 (10:43 am)
My point about DRM is that is not really useful, only the big studios can afford the better DRM systems, and they only protect for the initial x weeks where sales are at their best and at the same time the same DRM can cause legitamate users problems.I think adding DRM as an anti piracy measure is a pointless exercise for the indie studio, unless you expect to make like 75% of your sales in the first 2-4 weeks.
#8
...but what if you want to provide just enough protection to act as a deterrent to the casual and maybe not even conscious copy-send type of end-user? It is most likely that the end-users in my market will not even know, or care, but just want to share, and prompting them to register/pay would be a non-issue. Would using some of the e-distribution, compression and packaging ideas be sufficient? I still want to have an end user registration to activate process for updates, new releases, and cross-selling, etc.. believe me, if I could avoid the pain to implement DRM, I would, but I don't see an alternative yet...
02/21/2011 (11:54 am)
Point taken... so, the really big and successful games will have DRM cracked by the pro-hackers no matter what... that is most likely not me!...but what if you want to provide just enough protection to act as a deterrent to the casual and maybe not even conscious copy-send type of end-user? It is most likely that the end-users in my market will not even know, or care, but just want to share, and prompting them to register/pay would be a non-issue. Would using some of the e-distribution, compression and packaging ideas be sufficient? I still want to have an end user registration to activate process for updates, new releases, and cross-selling, etc.. believe me, if I could avoid the pain to implement DRM, I would, but I don't see an alternative yet...
#9
Also, make sure you arent wasting image channels. You may need to rewrite some shaders to fix this.
Reuse as much as possible. Think about using the same material for many objects, just laying out the UVs differently (ie one grassy clump image can be a material for many different grassy clump models). This also will keep your draw calls down.
To consolidate assets, place everything in a zip... you can use this fix to make it work in B3 (mounts the zip as a directory):
www.garagegames.com/community/forums/viewthread/123657
(the old tribes game used this technique for 'vol' files that you could open using winzip)
If you wanted to encrypt, you could use this resource (of course modified to your needs) to encrypt/decrypt your files (even the zip filestream above):
www.garagegames.com/community/resources/view/7253
02/21/2011 (11:55 am)
To address file size: For final release, do a LOT of research on the DDS image format, the different formats and compression. You can write scripts or develop actions/droplets in photoshop to process all of your working files to final DDS. This will help GREATLY in final release file size if done smartly. Google the Photoshop Nvidia DDS plugin for info on this. Also, make sure you arent wasting image channels. You may need to rewrite some shaders to fix this.
Reuse as much as possible. Think about using the same material for many objects, just laying out the UVs differently (ie one grassy clump image can be a material for many different grassy clump models). This also will keep your draw calls down.
To consolidate assets, place everything in a zip... you can use this fix to make it work in B3 (mounts the zip as a directory):
www.garagegames.com/community/forums/viewthread/123657
(the old tribes game used this technique for 'vol' files that you could open using winzip)
If you wanted to encrypt, you could use this resource (of course modified to your needs) to encrypt/decrypt your files (even the zip filestream above):
www.garagegames.com/community/resources/view/7253
#10
02/21/2011 (12:01 pm)
Thanks Jameson!!!! 
#11
I know i'm guilty, if I got something that is large and I'm downloading under 250 mbps, I dismiss and go find a faster link. Though good optimization can go a long way.
02/23/2011 (6:04 pm)
Where did you get the info on downloads being abandoned? I read something similar, but it had more to do with picking the proper hosting. Most of those abandoned downloads are probably when someone gets a download rate much slower than their average speed. This is subsequently due to lousy cheap hosting.I know i'm guilty, if I got something that is large and I'm downloading under 250 mbps, I dismiss and go find a faster link. Though good optimization can go a long way.
#12
Speed issues can be hard to detect, but you might have start and stop times and can guess from there. If it's not speed, people are simply impatient gits ;)
02/23/2011 (7:02 pm)
You can use server log analysis to figure out what people were doing. They hit your front page, they hit your download page, they start downloading, but the size downloaded is only a percentage of the full file size. If this happens a lot, it's usually one of two things: Your server has an issue delivering files, or people couldn't be arsed to wait.Speed issues can be hard to detect, but you might have start and stop times and can guess from there. If it's not speed, people are simply impatient gits ;)
#13
02/25/2011 (9:03 am)
I have tested my internet connection and it actually predicts a 33 minute download for such file size... I can optimize a bit, but my graphics are about as good as that will get. I can try the compression and decompression, that may help and I have a request out to Esellerate to see if they can do anything on their end... Also, I have looked at the Adobe UX process... they actually divide their programs into two parts, (1.39 GB app) and they use Akamai (http://www.akamai.com/)
#14
(Anything over 50MB sucks for UK, Australia and some parts of the US!)
02/25/2011 (12:02 pm)
Can you slim it down to a demo version with as little as possible? You'd basically have to make a second game with fewer features, but that could reduce it a bit.(Anything over 50MB sucks for UK, Australia and some parts of the US!)
#15
http://www.allegorithmic.com/?PAGE=PRODUCTS
the only problem is it most likely requires a lottery win to be able to purchase a license, like most sites that dont publish their prices on their site...
02/25/2011 (12:15 pm)
well... another option seeing as the bulk of game installs nowadays is the sheer size of the texture library, allegorithmics substance suite can really minimize the size footprinthttp://www.allegorithmic.com/?PAGE=PRODUCTS
the only problem is it most likely requires a lottery win to be able to purchase a license, like most sites that dont publish their prices on their site...
#16
The downside to this method is the fact that the user must be connected to the Internet in order to run the software, but there are a lot of benefits. For example I notice you make educational games. If your core market is teachers then when you have them create an account you can ask for what city, and state they are in. Also, maybe ask if they work at a Private, or Public institution. Maybe even ask what School District they work for (if Public Institution is selected). With this data you can see where your products are being used, and it will help you identify cities/states that are not using your product. This will help you target marketing of your products, and no personal identifying information is required from the user.
Now you will never be able to make anything Crack Proof! If someone really wants to they can circumvent any protection you put in place within hours of receiving a copy with a good Disassembler, and a working knowledge of the Assembly Language. Any protection you put in place is really only there to keep the majority of the people honest. Anyone determined to Crack, or obtain a Cracked copy will regardless of what you do.
03/06/2011 (2:07 am)
To address the issue with having people register this can be taken care of by requiring an account to access the software. Make it so that when the user first runs the program it asks for a Username, and Password (which they should have created prior to being able to download the software). When they hit enter the software connects to a server, and validates the Username, and the Password they entered. If everything is good then the software continues. If there is an error then it prompts for them to enter the credentials again with the option to create an account just like a lot of websites do (this can be helpful if you encourage your users to share the software freely). Once the user is validated then their credentials will be used to connect to the server to validate, and obtain updates automatically from then on. As a level of protect for the user you can store an MD5 Hash of the Password the user enters locally, and in your database. This way you only send the MD5 Hash to the server from the software. This helps protect the user's password.The downside to this method is the fact that the user must be connected to the Internet in order to run the software, but there are a lot of benefits. For example I notice you make educational games. If your core market is teachers then when you have them create an account you can ask for what city, and state they are in. Also, maybe ask if they work at a Private, or Public institution. Maybe even ask what School District they work for (if Public Institution is selected). With this data you can see where your products are being used, and it will help you identify cities/states that are not using your product. This will help you target marketing of your products, and no personal identifying information is required from the user.
Now you will never be able to make anything Crack Proof! If someone really wants to they can circumvent any protection you put in place within hours of receiving a copy with a good Disassembler, and a working knowledge of the Assembly Language. Any protection you put in place is really only there to keep the majority of the people honest. Anyone determined to Crack, or obtain a Cracked copy will regardless of what you do.
#17
But then again, I'm not developing anything for commercial use ... but it did reduce my ~180mb uncompressed game files into a ~98mb installer package.
03/15/2011 (1:48 pm)
Whats wrong with using the T3D toolbox to package your game ? I've used it to package my demo up for a class project - and seemed to work great, including having a direct X installed built into the process.But then again, I'm not developing anything for commercial use ... but it did reduce my ~180mb uncompressed game files into a ~98mb installer package.
Associate Steve Acaster
[YorkshireRifles.com]
1 and 2) T3D 1.1 final should have working zip compression - there's been posts for fixes on the forums - though I've not tried any of them. Also the stock installer will (fingers crossed) be fixed of reported issues.
2) But really it's just having an installer.
3) Pesonally ... I don't think DRM is worth bothering with - nothing a 12 year old script kiddy can't hack around in a few minutes. More hassle to developer and user than it's worth in hours implementing. I know that some distribution services like Steam require their own system implementing.
I've used genEPak for 1, 2 and to some extent. My art folder is about 300mb uncompressed and half that compressed. Currently genEPak is not easily patchable (I've no idea if Marton is going to make it so or not in the future).