I wish to combine two textures in a shader exactly the same as you would expect from a photoshop like program. That is, one texture ontop of another as if layered.
Both textures will have alpha, and the result output should also have the correct combined alpha.
However despite googleing for the formula, I am not able to get the correct colors out.
Using the referance here: (https://microsoft.github.io/Win2D/html/PremultipliedAlpha.htm) for example produces;
(note difference in grey 50% overlay)
The shader code is set in libgdx to use this as the blending;
context.setBlending(true,GL20.GL_SRC_ALPHA ,GL20.GL_ONE_MINUS_SRC_ALPHA);
I am fairly sure the mistake is in my fragment shaders blending (see code in image above), because if I pass the input texture (A) straight to gl_FragColor, then I get a pixel-perfect color match of A, including its alpha.
eg;
gl_FragColor = backdiffuse;
works fine.
=====
edit
Following advice, I have tried using a premulitiplied alpha instead.
context.setBlending(true,GL20.GL_ONE, GL20.GL_ONE_MINUS_SRC_ALPHA);
However, this produces wrong effects even when the shader is just set to output A
backdiffuse = texture2D(u_backSample2D, vTexCoord);
backdiffuse=backdiffuse*backdiffuse.a; //convert to premultiply
gl_FragColor = backdiffuse;
Am I setting libgdxs blending wrong? This is clearly too bright.
Solution (thanks to accepted answer);
backdiffuse.rgb *= backdiffuse.a; //premultiply source
v_backBackColor.rgb *= v_backBackColor.a; //premultiply source
finalCol.rgb = backdiffuse.rgb + (v_backBackColor.rgb * (1 - backdiffuse.a));
finalCol.a = backdiffuse.a + (v_backBackColor.a * (1.0 - backdiffuse.a));
2
Answers
If you want to use the premultiplied alpha equation, you must also draw the output to screen with the appropriate blend equation: GL_ONE, GL_ONE_MINUS_SRC_ALPHA.
And the destination alpha must also be multiplied by one minus source alpha. So:
If you want to use the more typical blend equation you are currently using, the equations in the shader would be more complicated. You can find them if you look up alpha compositing on Wikipedia. However, premultiplied alpha is a preferred format since it avoids dark or light fringing on antialiased edges of sprites.
Here is function that computes the "over" alpha compositing function (without pre-multiplied alpha).
This is only necessary if both colors are translucent. If you are always blending onto an opaque color, you can use a simpler function.