HLSL question. Trying to fix a bug in a postFX outline shader
by David Stocker · in Torque 3D Professional · 09/23/2010 (4:48 pm) · 3 replies
Hi All,
I’m trying to apply Matt Jolly’s full screen Sobel outline postFX shader on 1.1 beta 3 and I’m getting the following error at line 28 of outlineDetectP.hlsl.
"cannot implicitly convert from 'const float4' to 'sampler2D'"
This is happening at the following line of code:
float4 gbSample = prepassUncondition( prepassBuffer, uv );
prepassBuffer is a sampler2D object and it appears to my naïve eyes (as I don’t specifically know hlsl), that the constructor for float4 does not like having a sampler2D as the first parameter. I’m kind of green at hlsl, so does anyone have any tips about how I might work through this?
Thanks,
Dave
Edit: Fixed the markup in the hyperlink
I’m trying to apply Matt Jolly’s full screen Sobel outline postFX shader on 1.1 beta 3 and I’m getting the following error at line 28 of outlineDetectP.hlsl.
"cannot implicitly convert from 'const float4' to 'sampler2D'"
This is happening at the following line of code:
float4 gbSample = prepassUncondition( prepassBuffer, uv );
prepassBuffer is a sampler2D object and it appears to my naïve eyes (as I don’t specifically know hlsl), that the constructor for float4 does not like having a sampler2D as the first parameter. I’m kind of green at hlsl, so does anyone have any tips about how I might work through this?
Thanks,
Dave
Edit: Fixed the markup in the hyperlink
#2
or
(which was Matt Jolly's substitution during one of the original 1.0 betas and mentioned in this thread.
The full error is as follows:
I've googled Sampler2D objects and they seem to be a device for getting the color at a particular coordinate, based on the texture (and anything else?), while float4 seems to be a float primitive for defining color. What I don't seem to be able to do is to get the sampler2d info into the float4 format. Oddly however, the message points to a cast error in the other direction, as if the Sampler2D was getting its info from a float4.
I guess I'll be learning a lot about hlsl trying to figure this out.
09/23/2010 (8:57 pm)
It does not seem to matter whether you are using float4 gbSample = prepassUncondition(tex2D(prepassBuffer, uv));
or
float4 gbSample = prepassUncondition( prepassBuffer, uv );
(which was Matt Jolly's substitution during one of the original 1.0 betas and mentioned in this thread.
The full error is as follows:
Quote:GFXD3D9Shader::_compileShader - Error compiling shader: E_FAIL: An undetermined error occurres (80004005)
../game/shaders/common/postFx/outlineDetectP.hlsl(28,25): error X3017: 'autogenUncondition_55070f7a': cannot implicitly convert from 'const float4' to 'sampler2D'
I've googled Sampler2D objects and they seem to be a device for getting the color at a particular coordinate, based on the texture (and anything else?), while float4 seems to be a float primitive for defining color. What I don't seem to be able to do is to get the sampler2d info into the float4 format. Oddly however, the message points to a cast error in the other direction, as if the Sampler2D was getting its info from a float4.
I guess I'll be learning a lot about hlsl trying to figure this out.
#3
09/25/2010 (7:46 am)
I got it wroking. Matt Jolly's hlsl code is fine. I was a file editing error on my part. :O 
Torque 3D Owner Bryan Sawler
muteki corporation
Change that line to
float4 gbSample = prepassUncondition(tex2D(prepassBuffer, uv));