Additive Animation Blending

Westquote

Hi all,

Thanks for all your help so far in achieving the various goals I have for this project - this is a great community. 🙂

I am currently trying to figure out how to implement additive animation blending. If you're unfamiliar with the concept, the idea is similar to additive image blending, wherein the result of the blend is (a * alpha + b) instead of (a * alpha + b * (1 - alpha)). The result, in animation, is a reusable/cheap mechanism for implementing subtle animation effects like breathing or hand shake. Additionally, with blend shapes (aka FFD), additive blending allows more advanced handles for combining blend shapes for effects like facial animation and advanced character body/face customization.

The Uncharted team gave a talk about Additive and partial animations that I attended a while back - it's online now, and should be a good watch: http://www.gdcvault.com/play/1012300/Animation-and-Player-Control-in

So far, I haven't found anything in the C runtime that allows additive animation, but I do think I see a very easy/cheap set of changes that would allow it to be possible/efficient within the current runtime with no changes to the editor. All that would be needed is for a new function be added such as:

void spAnimation_mixEx (const spAnimation* self, struct spSkeleton* skeleton, float lastTime, float time, int loop, spEvent** events, int* eventsCount, float alpha, float targetAlpha) {
  int i, n = self->timelinesCount;

  if (loop && self->duration) {
    time = FMOD(time, self->duration);
    lastTime = FMOD(lastTime, self->duration);
  }

  for (i = 0; i < n; ++i)
    spTimeline_apply(self->timelines[i], skeleton, lastTime, time, events, eventsCount, alpha, targetAlpha);
}

Then, mix would become:

void spAnimation_mix (const spAnimation* self, spSkeleton* skeleton, float lastTime, float time, int loop, spEvent** events, int* eventsCount, float alpha) {
   spAnimation_mixEx(self, skeleton, lastTime, time, loop, events, eventsCount, alpha, 1.0f - alpha);
}

The last (and most intrusive) step would be to modify the _spTimelineVtable interface to accept the targetAlpha parameter on blend, and to instrument each of the sp*Timeline_apply() functions to use the parameter where appropriate. Skimming the source code, this would be Rotate, Translate, Scale, Color, FFD, and IK, and the actual changes would be extremely straightforward/safe/performant.

Anyway, I'm considering making this change on my local branch, but I thought I'd check in here first to see a) if it's already been planned/discussed, b) if it's already possible through some other means, or c) if there are problems with this implementation that others more familiar with the codebase might be able to see.

Let me know if you have any thoughts! 🙂

Cheers,

Tim A.

P.S. An interesting side-effect of this approach would be the ability to mix animations with pre-calculated final blend coefficients. That is, if you wanted the final blend of 3 anims to be 25%, 25%, and 50%, you could mix them in any order, providing 0.25, 0.25, and 0.5 for alpha, and 1 for targetAlpha. I have used a similar technique in the past in OpenGL graphics code for combining both additive blend and decal blend into a single unified blend mode for more efficient sprite batching (by reducing draw calls). A neat "follow-on feature".

Westquote

jpoag

Ok, I don't think this is a solution, but I have been looking over the code:

Spine does have a tracking system that allows you to play concurrent (even looping) tracks over each other. By default the tracks override each other, so if you're animating 'arm' in both tracks 1 and 0 then track 1 will specify the animation.

There is a little known parameter for a track called 'mix' which translates into the 'alpha' for mixing. Now, the alpha parameter might work like additive blending because I'm not seeing how the previous value is scaled back by (1-alpha). What I'm seeing is: bone->rotation += amount * alpha; which looks like (a * alpha + b) instead of (a * alpha + b*(1-alpha)). However, it's late so I could be missing something.

In general I like the idea of being able to specify how the animations are blended, but I would probably prefer a specific set of enumerated techniques {normal, additive, multiplicative, etc...}

BinaryCats

jpoag ha scritto
Ok, I don't think this is a solution, but I have been looking over the code:

Spine does have a tracking system that allows you to play concurrent (even looping) tracks over each other. By default the tracks override each other, so if you're animating 'arm' in both tracks 1 and 0 then track 1 will specify the animation.

There is a little known parameter for a track called 'mix' which translates into the 'alpha' for mixing. Now, the alpha parameter might work like additive blending because I'm not seeing how the previous value is scaled back by (1-alpha). What I'm seeing is: bone->rotation += amount * alpha; which looks like (a * alpha + b) instead of (a * alpha + b*(1-alpha)). However, it's late so I could be missing something.

In general I like the idea of being able to specify how the animations are blended, but I would probably prefer a specific set of enumerated techniques {normal, additive, multiplicative, etc...}

it was , under the impression that alpha does work like additive blending. this isn't documented though.

Westquote

jpoag ha scritto
There is a little known parameter for a track called 'mix' which translates into the 'alpha' for mixing. Now, the alpha parameter might work like additive blending because I'm not seeing how the previous value is scaled back by (1-alpha). What I'm seeing is: bone->rotation += amount * alpha; which looks like (a * alpha + b) instead of (a * alpha + b*(1-alpha)).

I had the exact same thought upon a quick skim, but upon (slightly) closer inspection of this line...

bone->x += (bone->data->x + self->frames[self->framesCount - 2] - bone->x) * alpha;

...I realized that this transforms as follows...

bone->x = (bone->x) + (bone->data->x * alpha) + (self->frames[self->framesCount - 2] * alpha) - (bone->x * alpha);

bone->x = (bone->data->x * alpha) + (self->frames[self->framesCount - 2] * alpha) + (bone->x - bone->x * alpha);

bone->x = (bone->data->x * alpha) + (self->frames[self->framesCount - 2] * alpha) + bone->x * (1 - 1 * alpha);

bone->x = (bone->data->x * alpha) + (self->frames[self->framesCount - 2] * alpha) + bone->x * (1 - alpha);

...such that it is just an efficient way of calculating a normal (non-additive) blend.

jpoag ha scritto
In general I like the idea of being able to specify how the animations are blended, but I would probably prefer a specific set of enumerated techniques {normal, additive, multiplicative, etc...}

I considered this, as well, but concluded that the only blends I've ever heard of being used in animation (even after Googling for it) are normal and additive. The 3D package Blender does offer both multiplicative and subtractive animation blends, suggesting they may be aware of some advanced use case? My 2 cents, though, is that normal+additive blends should handle 99% of practical use cases, and can be implemented within a single fixed-function (whereas multiplicative and subtractive would require conditional branching of some kind).

Cheers,

Tim A.

jpoag

Ok, so maybe not multiplicative, subtractive, etc... but I would still like to just specify the alpha and the mode and have to think about

Additive : { alpha, 1} and
Normal : {alpha, 1-alpha}

Or maybe use your code as-is and add an additional member to the track struct for blend type that defaults to current behavior.

I think it's a good idea, the main hurdle getting it official is that all of the runtimes need to support it. Since it's such a core feature you're really only looking at branching the C#, C, Lua, AS and maybe JS runtime cores. You have a good chance with modifying the track code because IIRC, it's not an editor feature, only runtime.

Best way to get it in is to fork a github repo, create the changes and send a pull request. I think there's some legal document you need to sign relinquishing copyright, but nothing significant.