Custom shaders/filters for spine-pixi-v8

Spinebot

For setting custom shaders or filters for Spine meshes in a pixi-spine environment, you would typically need to work with the WebGL or WebGPU context directly. Unfortunately, there isn't specific documentation from Esoteric Software for pixi-spine as it is maintained by a third party. However, you can apply custom shaders by modifying the shader code used by the Spine runtime. You might need to delve into the pixi-spine source code to understand how it handles shaders and then apply your custom shader logic accordingly. If you are looking to set a custom Depth (Z) value, you would likely need to modify the vertex shader to adjust the Z position of the vertices.

For more detailed guidance, you might want to explore the pixi-spine GitHub repository or reach out to the community there for specific examples and support.

Davide

benjaminhon

Hello! Pixi filters work with spine-pixi-v8.

See the basic examples with an AsciiFilter.

I haven't tried yet to create a custom filter, but mimicking the existing filters, I guess you can achieve your desired result.

benjaminhon

Ah yes it works, i think i was setting the uniforms wrong

benjaminhon

Hmmm I managed to get the filter working for the entire Spine container, however how do we handle the case for custom shaders for individual parts?

For instance:

Normal maps, lighting

For this we would need to be able to set the shaders for the individual meshes instead of the whole container

Davide

benjaminhon

Apologies for the delayed response; I've been away these past few days.
Currently, it’s not possible to modify the shader of individual attachments directly.

However, with a bit more effort, you can achieve the same effect by using a slot object, as demonstrated in the following example:

To do this, you can extract the individual texture and replace the corresponding attachment with a slot object. Since slot objects are standard Pixi elements, you can then easily apply a filter to them.

benjaminhon

Davide I see, thanks.

benjaminhon

Hi @Davide i am wondering as i was looking into the source code in the pixi spine v8 repo under

Spine.ts

readonly _slotsObject: Record<string, { slot: Slot, container: Container, followAttachmentTimeline: boolean } | null>

Do you think it will work if i iterate over the slots which has the pixijs Container and apply my normal filter map there?

In my filter, i pass in the original texture (the atlas) so i can remap the vTextureCoord to the original texture coordinate to sample the normal map.

Im not sure if theres more things that goes on behind the hood, but do you think this would work?

precision mediump float;

in vec2 vTextureCoord;

uniform vec4 uInputSize;
uniform vec4 uOutputFrame;
uniform sampler2D uTexture;
uniform sampler2D uNormalTexture;
uniform vec3 uLight; // x, y, intensity
uniform float uAmbient;

void main(void)
{
    // Sample the original texture using vTextureCoord
    vec4 color = texture2D(uTexture, vTextureCoord);

    if (color.a < 0.0001) {
        discard;
    }

    if(uLight[2] > 0.0) {
        // Sample the normal map (the UVs for uNormalTexture is not vTextureCoord, need to recalculate)
        // See https://github.com/pixijs/pixijs/issues/6266
        // See https://github.com/pixijs/pixijs/wiki/v5-Creating-filters#conversion-functions
        vec4 normalColor = texture2D(uNormalTexture, vTextureCoord * uInputSize.xy /  uOutputFrame.zw);
        
        // Convert normal map color from [0,1] to [-1,1] range
        vec3 normal = normalize(normalColor.rgb * 2.0 - 1.0);
        
        // Create light direction vector
        vec3 lightDir = normalize(vec3(uLight.xy, 1.0));
        
        // Calculate dot product between normal and light direction
        float lightIntensity = max(dot(normal, lightDir), 0.0);
        
        // Apply light intensity and user-defined light strength
        float finalLightIntensity = mix(uAmbient, 1.0, lightIntensity * uLight[2]);
        
        // Apply lighting to color
        color.rgb *= finalLightIntensity;
    }

    gl_FragColor = color;
}

benjaminhon

On further investigation it seems i cant get the containers of the original skin, it seems that only added slot objects via code have containers and the original skin is rendered internally as 1 entire ViewContainer instead

Davide

benjaminhon

As you've already experienced, it doesn't work. With Pixi v8, we went low-level to improve performance. On the flip side, this means there's less room for customization.

However, there may be an alternative approach. Have you looked at the tint black batcher code?
Ideally, you could use it as a base, define your own shader, and implement further customizations.

benjaminhon

i guess ill wait for future updates 😃 that support custom filters, currently im working around it by using LSB stenganogragy for my material maps and using a filter on the entire container. Though its only sufficient for material maps not normal maps

FloodGames

Can someone please make a demo using these 4 files?

EsotericSoftware/spine-runtimestree/4.2/spine-ts/spine-pixi-v8/src/darktint

Does it look similar to this beautiful shadow?

What can we do to make it as much as possible like this?

Davide

FloodGames

It's overkill to create a dedicated batcher for that as a first step.

There's a shadow filter in pixi-filters that you can try.
If that doesn't suit you, you can create a custom filter by adapting the shader you linked.
Only if filter performance doesn't satisfy you should you create a dedicated batcher for that.

FloodGames

Thank you for your response!

Any chance you could add this example,
https://pixijs.io/examples-v5/#/filters-advanced/pixie-shadow-filter.js

To the latest version please?

Dropshadow from pixi.filters seems something different.
https://filters.pixijs.download/main/examples/index.html?enabled=DropShadowFilter

Versus

(drop shadow vs cast shadow)

FloodGames

Haha, Grok got the shadow shader 'working' on a coin, looking very broken:

FloodGames

So I got the following animation working (x of the projection shadow is a little off):

It creates a renderTexture of the spine and uses that plus a shadow of that...?

Little bit wonky code (I don't understand many parts regarding the shader and rendering parts):

`<html>
  <head>
    <meta charset="UTF-8" />
    <title>spine-pixi-v8 with Shadow Fix</title>
    <script src="https://cdn.jsdelivr.net/npm/pixi.js@8.4.1/dist/pixi.js"></script>
    <script src="../dist/iife/spine-pixi-v8.js"></script>
    <script src="https://cdn.jsdelivr.net/npm/lil-gui@0.20.0/dist/lil-gui.umd.min.js"></script>
    <link href="https://cdn.jsdelivr.net/npm/lil-gui@0.20.0/dist/lil-gui.min.css" rel="stylesheet">
    <link rel="stylesheet" href="../../index.css">
  </head>

  <body>
    <script>
      (async function () {
        var app = new PIXI.Application();
        await app.init({
          width: window.innerWidth,
          height: window.innerHeight,
          resolution: window.devicePixelRatio || 1,
          autoDensity: true,
          resizeTo: window,
          backgroundColor: 0xffffff, // White for better contrast
          preference: 'webgl', // Force WebGL
          hello: true,
        })
        document.body.appendChild(app.canvas);

        // Load assets
        PIXI.Assets.add({alias: "coinData", src: "/assets/coin-pro.json"});
        PIXI.Assets.add({alias: "coinAtlas", src: "/assets/coin-pma.atlas"});
        await PIXI.Assets.load(["coinData", "coinAtlas"]);

        // Create Spine object
        const coin = spine.Spine.from({
          skeleton: "coinData",
          atlas: "coinAtlas",
          scale: 1,
        });
        coin.autoUpdate = false;
        coin.state.data.defaultMix = 0.2;
        coin.state.setAnimation(0, "animation", true);
        coin.update(0); // Initial update for bounds

        // Compute base Y from local bounds
        const localBounds = coin.getLocalBounds();
        const baseLocalY = localBounds.y + localBounds.height;
        const padding = 300;
        const rtWidth = localBounds.width + padding * 2;
        const rtHeight = localBounds.height + padding * 2; // Extra for shadow downward

        // Create RenderTexture for composing the Spine render
        const rt = PIXI.RenderTexture.create({ width: rtWidth, height: rtHeight });

        // Create a Sprite to display the composed texture
        const composedSprite = new PIXI.Sprite(rt);
        composedSprite.anchor.set(0.5);

        // Hide the original Spine object
        coin.visible = false;
        app.stage.addChild(coin); // Still add to stage for transform calculations
        app.stage.addChild(composedSprite);

        // Vertex shader with mediump precision
        const myVertex = `
          #version 300 es
          precision mediump float;

          in vec2 aPosition;

          uniform vec4 uInputSize;
          uniform vec4 uOutputFrame;
          uniform vec4 uOutputTexture;

          out vec2 vTextureCoord;

          vec4 filterVertexPosition(void) {
              vec2 position = aPosition * uOutputFrame.zw + uOutputFrame.xy;
              position.x = position.x * (2.0 / uOutputTexture.x) - 1.0;
              position.y = position.y * (2.0 * uOutputTexture.z / uOutputTexture.y) - uOutputTexture.z;
              return vec4(position, 0.0, 1.0);
          }

          vec2 filterTextureCoord(void) {
              return aPosition * (uOutputFrame.zw * uInputSize.zw);
          }

          void main(void) {
              gl_Position = filterVertexPosition();
              vTextureCoord = filterTextureCoord();
          }
        `;

        // Fragment shader
        const myFragment = `
          #version 300 es
          precision mediump float;

          in vec2 vTextureCoord;

          uniform sampler2D uSampler;
          uniform vec4 uInputSize;
          uniform vec4 uOutputFrame;
          uniform vec2 shadowDirection;
          uniform float floorY;

          out vec4 outColor;

          void main(void) {
              vec2 screenCoord = vTextureCoord * uInputSize.xy + uOutputFrame.xy;
              vec2 shadow;
              float paramY = (screenCoord.y - floorY) / shadowDirection.y;
              shadow.y = paramY + floorY;
              shadow.x = screenCoord.x + paramY * shadowDirection.x;
              vec2 bodyFilterCoord = (shadow - uOutputFrame.xy) * uInputSize.zw;

              vec4 originalColor = texture(uSampler, vTextureCoord);
              vec4 shadowColor = texture(uSampler, bodyFilterCoord);
              shadowColor.rgb = vec3(0.0);
              shadowColor.a *= 0.8;

              outColor = originalColor + shadowColor * (1.0 - originalColor.a);
          }
        `;

        const glProgram = new PIXI.GlProgram({
          vertex: myVertex,
          fragment: myFragment,
        });

        const shadowUniforms = {
          shadowDirection: { value: [-0.3, -0.7], type: 'vec2<f32>' },
          floorY: { value: 0.0, type: 'f32' },
        };

        const filter = new PIXI.Filter({
          glProgram,
          resources: {
            shadowUniforms,
          },
        });
        filter.padding = 300;

        composedSprite.filters = [filter];

        // Function to update positions
        function updatePositions() {
          const centerX = window.innerWidth / 2;
          const centerY = window.innerHeight / 2;
          coin.x = centerX;
          coin.y = centerY;
          composedSprite.x = centerX;
          composedSprite.y = centerY;
        }

        updatePositions();

        // Add resize listener
        window.addEventListener('resize', updatePositions);

        app.ticker.add((ticker) => {
            coin.update(ticker.deltaMS / 1000);

            // Render Spine to RenderTexture with offset to center it
            const offsetX = -localBounds.x - localBounds.width / 2 + rtWidth / 2;
            const offsetY = -localBounds.y - localBounds.height / 2 + rtHeight / 2;
            const matrix = new PIXI.Matrix().translate(offsetX, offsetY);
            app.renderer.render(coin, {
              renderTexture: rt,
              clear: true,
              transform: matrix
            });
            coin.x=400
            coin.y=300

            // Update floorY using global position
            filter.resources.shadowUniforms.uniforms.floorY = coin.toGlobal(new PIXI.Point(0, baseLocalY)).y;
        });
      })();
    </script>
  </body>
</html>
`

Is this the best approach...?

FloodGames

It seems possible to get something like this, but the way I am doing it seems very poor (active Spine + a copy RenderTexture of the Spine with a shadow filter on it):