Huh cool: extracting alpha w/o alpha channel
by Orion Elenzil · in Torque Game Engine · 06/13/2008 (4:47 pm) · 10 replies
Funny,
we went to some trouble a bit ago to add an alpha buffer to our TGE-based app
so that in a special mode we could take screenshots with high-quality transparency
instead of green-screening.
then today i stumbled on TSShapeInstance::snapshot(),
which has a pretty cool technique for extracting alpha channel information
without actually having an alpha buffer:
it takes advantage of the fact that we know the blending equation
and can write the following equations:
when an object is rendered on top of a black background,
the value of a given pixel is alpha * color.
when an object is rendered on top of a black background,
the value of a given pixel is alpha * color + (1 - alpha) * 255.
so using algebra,
alpha = [255 - (pixelViaWhiteBG - pixelViaBlackBG)]/255.
pretty cool,
and in fact i think produces higher-quality alpha than using an alpha buffer for two reasons:
1. the alpha buffer does not get anti-aliased, so the edges of polygons jump from 100% to 0% alpha. i worked around this by rendering the images larger-than-life and then shrinking them.
2. the alpha stores the last value of alpha written to it, so if you have something opaque underneath something transparent, you get a transparent alpha value. i worked around this by multiplying the alpha value times a large number (zero stays zero, everything else tends to opaque)
we went to some trouble a bit ago to add an alpha buffer to our TGE-based app
so that in a special mode we could take screenshots with high-quality transparency
instead of green-screening.
then today i stumbled on TSShapeInstance::snapshot(),
which has a pretty cool technique for extracting alpha channel information
without actually having an alpha buffer:
it takes advantage of the fact that we know the blending equation
and can write the following equations:
when an object is rendered on top of a black background,
the value of a given pixel is alpha * color.
when an object is rendered on top of a black background,
the value of a given pixel is alpha * color + (1 - alpha) * 255.
so using algebra,
alpha = [255 - (pixelViaWhiteBG - pixelViaBlackBG)]/255.
pretty cool,
and in fact i think produces higher-quality alpha than using an alpha buffer for two reasons:
1. the alpha buffer does not get anti-aliased, so the edges of polygons jump from 100% to 0% alpha. i worked around this by rendering the images larger-than-life and then shrinking them.
2. the alpha stores the last value of alpha written to it, so if you have something opaque underneath something transparent, you get a transparent alpha value. i worked around this by multiplying the alpha value times a large number (zero stays zero, everything else tends to opaque)
About the author
#2
06/20/2008 (8:55 am)
Very cool!
#3
06/20/2008 (1:50 pm)
I'm sure this could also be done using imageMagick or a similar image toolset (or photoshop) as well.
#4
i used imageMagick to automate this stuff, and in summary:
it works great, but you've got to get right in there with imageMagick.
here's my procedure:
part one,
create an image that is the alpha channel, where white = opaque and black = transparent:
src_whiteBG.png - the scene rendered on white
src_blackBG.png - the scene rendered on black
part two,
use the alpha channel and the original black-backgrounded image to create an image which has both the original color info and the alpha.
many thanks to Mike Plotz for showing me how to use imageMagick's -fx operator here.
part two, step one:
reconstructing the original RGB values of the scene before they were layered on top of a black background.
remember that when layering over black, pixelResult = pixelOriginal * alpha, so pixelOriginal = pixelResult / alpha.
i originally tried to use imagemagick's -compose divide operator,
but the results were not right. i think it may have been dividing by the average of the entire image rather than per-pixel.
anyhow, Mike Plotz and the -fx operator to the rescue, which lets you do arbitrary math on a per-pixel basis.
let's break that sucker down:
convert src_blackBG.png alphaChannel.png - launch convert with two images
-channel RGB - limit our operation to the RGB channels
-fx "u/(v.r + 0.001)" - evaluate the given expression for each pixel, where "u" is the pixel from src_blackBG.png and "v" is the pixel from alphaChannel.png. We add a bit to avoid dividing by zero.
foo.png - output to foo.png
part two, step two:
merge out calculated alphaChannel back in as the alpha channel.
the basic command here is convert rgbImage.png alphaChannel.png -compose Copy_Opacity -composite output.png,
but i'm getting ambitious and am going to combine that with the previous command:
let's break that sucker down:
everything up to before the second "alphaChannel.png" - this is the previous command but instead of outputting to foo.png, the output becomes the only item in imageMagick's image stack.
alphaChannel.png - (the second one) - add alphaChannel.png to the image stack
-compose Copy_Opacity -composite - replace the opacity of the first image in the stack with that of the second.
output.png32 - output as a 32-bit png. i found this necessary for photoshop compatibility.
mv output.png32 output.png - rename back to a regular png.
images to follow.
edit: added -colorSpace gray to the alphaChannel generation.
11/10/2008 (2:27 pm)
Just to follow up a bit here.i used imageMagick to automate this stuff, and in summary:
it works great, but you've got to get right in there with imageMagick.
here's my procedure:
part one,
create an image that is the alpha channel, where white = opaque and black = transparent:
src_whiteBG.png - the scene rendered on white
src_blackBG.png - the scene rendered on black
convert src_blackBG.png src_whiteBG.png -colorSpace gray -compose minus -composite -negate alphaChannel.png
part two,
use the alpha channel and the original black-backgrounded image to create an image which has both the original color info and the alpha.
many thanks to Mike Plotz for showing me how to use imageMagick's -fx operator here.
part two, step one:
reconstructing the original RGB values of the scene before they were layered on top of a black background.
remember that when layering over black, pixelResult = pixelOriginal * alpha, so pixelOriginal = pixelResult / alpha.
i originally tried to use imagemagick's -compose divide operator,
but the results were not right. i think it may have been dividing by the average of the entire image rather than per-pixel.
anyhow, Mike Plotz and the -fx operator to the rescue, which lets you do arbitrary math on a per-pixel basis.
convert src_blackBG.png alphaChannel.png -channel RGB -fx "u/(v.r + 0.001)" foo.png
let's break that sucker down:
convert src_blackBG.png alphaChannel.png - launch convert with two images
-channel RGB - limit our operation to the RGB channels
-fx "u/(v.r + 0.001)" - evaluate the given expression for each pixel, where "u" is the pixel from src_blackBG.png and "v" is the pixel from alphaChannel.png. We add a bit to avoid dividing by zero.
foo.png - output to foo.png
part two, step two:
merge out calculated alphaChannel back in as the alpha channel.
the basic command here is convert rgbImage.png alphaChannel.png -compose Copy_Opacity -composite output.png,
but i'm getting ambitious and am going to combine that with the previous command:
convert src_blackBG.png alphaChannel.png -channel RGB -fx "u/(v.r + 0.001)" alphaChannel.png -compose Copy_Opacity -composite output.png32 ; mv output.png32 output.png
let's break that sucker down:
everything up to before the second "alphaChannel.png" - this is the previous command but instead of outputting to foo.png, the output becomes the only item in imageMagick's image stack.
alphaChannel.png - (the second one) - add alphaChannel.png to the image stack
-compose Copy_Opacity -composite - replace the opacity of the first image in the stack with that of the second.
output.png32 - output as a 32-bit png. i found this necessary for photoshop compatibility.
mv output.png32 output.png - rename back to a regular png.
images to follow.
edit: added -colorSpace gray to the alphaChannel generation.
#5
the source scene, after being rendered on top of white:
the source scene, after being rendered on top of black:
the alpha channel, calculated from the two source images:
the "original" pixels values in the scene, calculated from src_blackBG.png and the alpha channel:
(note this is only an intermediate result)
the "original" scene pixel values, independent of background and with proper alpha:
(this is the final result)
and just for verification, here are some reconstructions:
output.png layered on white: (compare to src_whiteBG.png)
output.png layered on black: (compare to src_blackBG.png)
output.png layered on blue:
again, thanks to mike plotz for help with imagemagick's -fx.
11/10/2008 (2:42 pm)
Here are some images to accompany the preceding post.the source scene, after being rendered on top of white:
the source scene, after being rendered on top of black:
the alpha channel, calculated from the two source images:
the "original" pixels values in the scene, calculated from src_blackBG.png and the alpha channel:
(note this is only an intermediate result)
the "original" scene pixel values, independent of background and with proper alpha:
(this is the final result)
and just for verification, here are some reconstructions:
output.png layered on white: (compare to src_whiteBG.png)
output.png layered on black: (compare to src_blackBG.png)
output.png layered on blue:
again, thanks to mike plotz for help with imagemagick's -fx.
#6
11/25/2008 (7:04 am)
The coolness factor of this just multiplied!
#7
there's been some queries from folks who don't have access to this one.
06/09/2009 (9:52 am)
david / GG, any chance this thread could be moved into a public forum ?there's been some queries from folks who don't have access to this one.
#8
"the alpha stores the last value of alpha written to it, so if you have something opaque underneath something transparent, you get a transparent alpha value. i worked around this by multiplying the alpha value times a large number (zero stays zero, everything else tends to opaque)"
??
10/31/2009 (4:09 pm)
Orion, I've been doing some work with snapshot() (I stumbled on it by accident actually, and it's pretty much EXACTLY what I needed), and I was noticing that the transparency seems to be wayyyy too high, even for shapes that shouldn't be transparent at all. Is this what you meant by"the alpha stores the last value of alpha written to it, so if you have something opaque underneath something transparent, you get a transparent alpha value. i worked around this by multiplying the alpha value times a large number (zero stays zero, everything else tends to opaque)"
??
#9
hm, strange that you're getting transparency on visually opaque objects with the snapshot() technique. perhaps there's a flaw in the algo ?
the comment you quote is in reference to a different approach, in which you add a fourth channel to the render buffer and each pixel writes out alpha in addition to rg&b. that approach has some flaws compared to the method of snapshot() or the imagemagick approach above.
10/31/2009 (6:38 pm)
hey elfprince -hm, strange that you're getting transparency on visually opaque objects with the snapshot() technique. perhaps there's a flaw in the algo ?
the comment you quote is in reference to a different approach, in which you add a fourth channel to the render buffer and each pixel writes out alpha in addition to rg&b. that approach has some flaws compared to the method of snapshot() or the imagemagick approach above.
#10
11/01/2009 (3:39 am)
I'm using it to generate a texture for a GUI control based on GuiBitmapCtrl, for use in my inventory system (so that I can have 3d object previews without continuously rerendering them), and even the objects that render completely opaque in-game end up looking like they're around 30% - 40% opacity. I spent a while looking through the code that combines the white and black background renders and couldn't find any obvious problems, though I'm also no expert on image manipulation techniques. 
Torque Owner Ed Zavada