zlib and libpng -- too old?
by Tim Scheiman · in Torque Game Builder · 01/11/2012 (5:23 pm) · 6 replies
Hey folks,
Digging through the source trying to debug and optimize something and I noticed that the versions of these two in use in T2D are well... really old.
libpng 1.0.12 June 2001
zlib 1.1.3 July 1998 (!)
BOTH libraries have updates as recently as 2010, and both also tout huge performance gains and security fixes in releases as far back as 2003...
zlib's 2003 release had some folks talking about 30% increase in png decompression speeds!
So:
Why is T2D still compiling against these ancient versions?
How much work would be involved getting around all the deprecated junk in order to bring the engine into the modern era here?
Has anyone done it already? (I saw some talk of updating one or the other of these over time in the forums, but can't find anything substantive.)
Also, will it help? (png load times)
Thanks for any thoughts anyone has.
-Tim
Digging through the source trying to debug and optimize something and I noticed that the versions of these two in use in T2D are well... really old.
libpng 1.0.12 June 2001
zlib 1.1.3 July 1998 (!)
BOTH libraries have updates as recently as 2010, and both also tout huge performance gains and security fixes in releases as far back as 2003...
zlib's 2003 release had some folks talking about 30% increase in png decompression speeds!
So:
Why is T2D still compiling against these ancient versions?
How much work would be involved getting around all the deprecated junk in order to bring the engine into the modern era here?
Has anyone done it already? (I saw some talk of updating one or the other of these over time in the forums, but can't find anything substantive.)
Also, will it help? (png load times)
Thanks for any thoughts anyone has.
-Tim
#2
I'll try to report back with some before and after png loading data later on... just in case anyone else wonders about this in the future.
01/12/2012 (10:59 am)
Nifty. I'll try to report back with some before and after png loading data later on... just in case anyone else wonders about this in the future.
#3
libpng version 1.2.46 - July 9, 2011
zlib version 1.2.5, April 19th, 2010
my test case made 100 datablocks of the same 1024 x 768 image, and then made 100 t2dStaticSprites in the scene with each of those datablocks.
I ran the test both with preloading ON and OFF in the datablocks.
All times are an average of three tries (I know more would be more useful, but I got tired of it.) :)
Preloading off:
Original DB Creation Time: 6109.3 ms
New DB Creation Time: 5813.3 ms
Original sprites creation time: 7565.3 ms
New sprites creation time: 6741.3
Adding these together gives us a shift from 13.67 seconds to 12.55 seconds -- 8.2% faster.
Preloading on:
Original datablocks: 7738.7 ms
New datablocks: 6954.7
Sprite creation time was unchanged (only 26.7 ms!)
Datablock creation time was 10% faster.
So there ya go. Not massive gains or anything, but worth having a look at when you're looking to optimize. Now lets just hope I didn't break anything...
01/12/2012 (5:49 pm)
OK, compiled against these:libpng version 1.2.46 - July 9, 2011
zlib version 1.2.5, April 19th, 2010
my test case made 100 datablocks of the same 1024 x 768 image, and then made 100 t2dStaticSprites in the scene with each of those datablocks.
I ran the test both with preloading ON and OFF in the datablocks.
All times are an average of three tries (I know more would be more useful, but I got tired of it.) :)
Preloading off:
Original DB Creation Time: 6109.3 ms
New DB Creation Time: 5813.3 ms
Original sprites creation time: 7565.3 ms
New sprites creation time: 6741.3
Adding these together gives us a shift from 13.67 seconds to 12.55 seconds -- 8.2% faster.
Preloading on:
Original datablocks: 7738.7 ms
New datablocks: 6954.7
Sprite creation time was unchanged (only 26.7 ms!)
Datablock creation time was 10% faster.
So there ya go. Not massive gains or anything, but worth having a look at when you're looking to optimize. Now lets just hope I didn't break anything...
#4
01/14/2012 (8:13 am)
That's very cool. I'll keep this in mind if I have performance issues and need to optimize. Someone should create a resource with optimization tips I bet it would help many and grow over time.
#5
Aside from some simple file structure changes it wasn't too bad to add them in. libpng has changed a bit, and the optimization switches were removed a few versions ago. I ended up having to comment out a couple sections of dgl/bitmapPng.cc to get it to compile:
Around line 144:
#if defined(PNG_LIBPNG_VER) && (PNG_LIBPNG_VER >= 10200)
png_uint_32 mask, flags;
flags = png_get_asm_flags(png_ptr);
mask = png_get_asm_flagmask(PNG_SELECT_READ | PNG_SELECT_WRITE);
png_set_asm_flags(png_ptr, flags | mask);
#endif
and around line 331:
// Set the compression level, image filters, and compression strategy...
png_ptr->flags |= PNG_FLAG_ZLIB_CUSTOM_STRATEGY;
png_ptr->zlib_strategy = strategy;
- Straight commenting them out seems to work - the engine still compile and runs games, though it probably bears looking into a bit more by someone more familiar with C++/engine changes :) I may try to look into it more when I've got more free time!
10/09/2012 (4:52 am)
I tried updating both of these to the most recent versions - zlib 1.2.7 and libpng 1.5.13 into T2D 1.7.6.Aside from some simple file structure changes it wasn't too bad to add them in. libpng has changed a bit, and the optimization switches were removed a few versions ago. I ended up having to comment out a couple sections of dgl/bitmapPng.cc to get it to compile:
Around line 144:
#if defined(PNG_LIBPNG_VER) && (PNG_LIBPNG_VER >= 10200)
png_uint_32 mask, flags;
flags = png_get_asm_flags(png_ptr);
mask = png_get_asm_flagmask(PNG_SELECT_READ | PNG_SELECT_WRITE);
png_set_asm_flags(png_ptr, flags | mask);
#endif
and around line 331:
// Set the compression level, image filters, and compression strategy...
png_ptr->flags |= PNG_FLAG_ZLIB_CUSTOM_STRATEGY;
png_ptr->zlib_strategy = strategy;
- Straight commenting them out seems to work - the engine still compile and runs games, though it probably bears looking into a bit more by someone more familiar with C++/engine changes :) I may try to look into it more when I've got more free time!
#6
Updating zlib is as simple as deleting all of the files in the ../lib/zlib/ folder, removing the files from the zlib project in VStudio, unzipping the new zlib files into the ../lib/zlib/ folder and adding them to the zlib project in VStudio.
Updating libpng takes a little more effort. In addition to the steps above, you need to:
1). Find and comment out the
2). Also in bitmapPng.cc, find these two lines:
and replace them with:
Recompile and off you go (hopefully!).
Another recommendation for everyone: get rid of lungif and .gif support altogether. T2D's version of lungif is woefully outdated. Last I checked it's still v.4.0 which was released in 1998 and doesn't support compression (so no writing of GIFs) due to the LZW patent fiasco going on at the time.
12/22/2012 (8:11 pm)
I just spent a few minutes updating to the newest zlib (v.1.2.7) and libpng (v.1.5.13). I'd recommend everyone doing this because T2D's versions of both are very out of date and there have been numerous security and performance-related updates since then.Updating zlib is as simple as deleting all of the files in the ../lib/zlib/ folder, removing the files from the zlib project in VStudio, unzipping the new zlib files into the ../lib/zlib/ folder and adding them to the zlib project in VStudio.
Updating libpng takes a little more effort. In addition to the steps above, you need to:
1). Find and comment out the
#if defined(PNG_LIBPNG_VER) && (PNG_LIBPNG_VER >= 10200)block in bitmapPng.cc. It's just below a line that reads:
// Enable optimizations if appropriate.
2). Also in bitmapPng.cc, find these two lines:
png_ptr->flags |= PNG_FLAG_ZLIB_CUSTOM_STRATEGY; png_ptr->zlib_strategy = strategy;
and replace them with:
png_set_compression_strategy(png_ptr, strategy);
Recompile and off you go (hopefully!).
Another recommendation for everyone: get rid of lungif and .gif support altogether. T2D's version of lungif is woefully outdated. Last I checked it's still v.4.0 which was released in 1998 and doesn't support compression (so no writing of GIFs) due to the LZW patent fiasco going on at the time.
Torque Owner Ben Sparks - Warspawn
StarOrdered Games
edit: just tried libpng, compiled and built without a hitch. no idea if there is any difference in performance.
edit: ok, tried zlib as well. it also worked, tho there are new / different files so you need to just remove all the .c and .h files from the project and add the new ones. it built and linked fine however. also, no idea if it makes any difference.