Animation Doesn't Change on Frame SetAnimation is called

Nate · Aug 19, 2023

ExNull-Tyelor Yes these are the events I'm referring to, as that's what your documentation calls them.

It is, but there is another type of "event" in the form of AnimationState and TrackEntry listeners (sometimes called callbacks), so I just wanted to be sure we're on the same page.

ExNull-Tyelor This means that say we get an event that tells us we should change animations at a specific point, call it the "swap" event, our listener could listen for "swap" and then call SetAnimation at this point, but because Apply was already called the AnimationState fires the Start callback immediately on SetAnimation, however the AnimationState doesn't change until the following frame as you pointed out, and thus doesn't fire off it's events that are keyed to the first frame of the animation: despite the fact that the AnimationState's Start callback is fired which leads to undesirable single frame delay between SetAnimation and the AnimationState being applied and it's events being fired.

start is fired when an animation is set as the current entry. That happens in setAnimation, addAnimation (if the track is empty), or update. It can't happen in apply. Becoming the current entry is separate from the skeleton being posed and events being fired, as you described correctly, which happens in apply.

ExNull-Tyelor The behavior I expected was that SetAnimation would immediately apply the AnimationState on that frame, so that the AnimationState's Start callback is fired, and then the AnimationState is applied: thus firing any events keyed to frame 0 in the callstack for SetAnimation. Or that the AnimationState's Start callback listeners would be fired the following frame immediately before automatically applying the AnimationState, but neither seems to be the case. To be honest to me, it seems like a bug that the AnimationState's Start callback is fired the frame before the AnimationState is actually applied

It doesn't work as you described for a number of reasons. Setting up animations and posing the skeleton need to be separate. You may want to configure a TrackEntry before applying the animations. You may have multiple animations to set up before you want to apply them. Applying an animation is one of the more expensive operations, often requiring more CPU effort than updating the world transforms for all the bones in a skeleton. Further, the AnimationState has no knowledge of the skeletons it may be applied to. As mentioned earlier, a single AnimationState can be applied to pose any number of skeletons.

The start callback fires when a track entry is made the current entry. I suppose it could be fired the first time a track entry is applied, but I'm not sure that would be better. Your use case may benefit, but likely other use cases would suffer. At the very least, it would complicate the understanding of the event to "the first apply after the track entry is made the current entry". If applying to multiple skeletons, it would only happen the first time, which may seem surprising.

It's simplest to fire start when the track entry becomes the current entry rather than tie start to applying the AnimationState. Applying is truly a separate operation and I'm not convinced tying start to apply improves the situation. I'm always open to being convinced of course, on this or anything else!

Instead of using start, you could use an event key on frame zero. That would only be fired when the animation is first applied (and on subsequent loops, but you can ignore those by checking TrackEntry isComplete).

Another option, one that is better as it doesn't require setting keys for every animation, could be to check the AnimationState track entries before it is applied. If any have a trackTime of zero, it means this will be the first time the track entry is being applied.

ExNull-Tyelor It sounds like you should wait one frame before setting the new animation in response to the event. Do that on the next frame in a callback that happens before apply.

This would seem to be incredibly tedious to implement in a way that isn't just tossing coroutines everywhere that we set animations (which is everywhere, as it's an animation-event based game).

Possibly the above ideas removes your reliance on start and lets you run code just before the first time an animation is applied. Otherwise, you could keep using start and try something like this:

// Nuke any queued animations.
TrackEntry current = state.getCurrent();
if (current != null) state.clearNext(current);
// Instead of setAnimation, use addAnimation so it will become the current 
// animation during the next AnimationState update (next game frame).
state.addAnimation(0, "name", true, 0.0001); // or -999

EExNull-Tyelor · Aug 19, 2023

Nate The start callback fires when a track entry is made the current entry. I suppose it could be fired the first time a track entry is applied, but I'm not sure that would be better. Your use case may benefit, but likely other use cases would suffer. At the very least, it would complicate the understanding of the event to "the first apply after the track entry is made the current entry". If applying to multiple skeletons, it would only happen the first time, which may seem surprising.

Thanks for the reply, and explanation for why it works the way it does. I wouldn't want to, nor try to convince you, to change the runtimes to handle just my use-case: especially if it broke other people's use-case for the AnimationState's Start callback logic. I agree if one was expecting the Start callback to fire multiple times when applying an AnimationState to multiple skeletons how confusing that would be, and likely difficult to handle that for each of those skeleton objects independently otherwise without some extra logic...

Nate Another option, one that is better as it doesn't require setting keys for every animation, could be to check the AnimationState track entries before it is applied. If any have a trackTime of zero, it means this will be the first time the track entry is being applied.

This seems like a good candidate for our use-case, as we could create logic in our update loop that fires a delegate for the logic we're using when we detect the trackTime 0 before updating the skeleton instead of subscribing to AnimationState's Start callback and calling the logic there. I'm off next week, but I'll be sure to try it out the following week and report back with my findings. This solution does sound promising though.

Thanks again for the response and explanations! It does make a lot more sense knowing why Start is fired when it is, especially the use case where someone is applying the AnimationState to many skeletons at the same time.

EExNull-Tyelor · Aug 29, 2023

Nate Another option, one that is better as it doesn't require setting keys for every animation, could be to check the AnimationState track entries before it is applied. If any have a trackTime of zero, it means this will be the first time the track entry is being applied.

Reporting back to say that this solution almost works, but doing this means that any animations added to the AnimationState, rather than being set to the animation state (i.e. when using AddAnimation and not SetAnimation) the added animations then won't trigger our Start Animation logic, since the TrackTime isn't 0 for those sequenced animations

Nate · Aug 29, 2023

Animations that are queued are not started because they have not yet been made the current track entry. The trackTime is 0 for those entries, but it won't start increasing until they are the current entry.

I don't understand what event you want from queued animations. You can walk the linked list of queued animations using TrackEntry next, but you can't use trackTime to know if this is the first frame they have been in the queued list, because it will be 0 until the entry is current. You could track that other ways, I suppose, but this does not feel like the most straightforward solution.

Stepping back and thinking about it from a high level with my limited understanding of your application, it seems you are setting animations, then rendering a frame so you see the last frame in the old animations, then you want events to happen for the newly set animations. This is causing issues for various reasons.

Why don't you render the frame first, so you see the last frame of your animations, then set your new animations and execute the logic for those? This way you don't need to delay anything a frame and don't need to use events or check if this is the first frame an animation will be applied.

EExNull-Tyelor · Aug 30, 2023

Nate Animations that are queued are not started because they have not yet been made the current track entry. The trackTime is 0 for those entries, but it won't start increasing until they are the current entry.

Sorry I think I must be bad at explaining myself and I think you misunderstood me. I mean that when a queued animation is set as the current track entry the trackTime is not 0 for that new animation. For example if I use SetAnimation followed by AddAnimation, when checking if the CurrentTrack's TrackTime is 0 in the update loop before manually updating the Skeleton, the animation in SetAnimation shows a TrackTime of 0. Then later once that animation finishes, I expect the CurrentTrack to change to the one queued with AddAnimation where it now would have a TrackTime of 0, but this isn't the case. Conversely the AnimationState's Start callback listener is called for both of these animations as expected.

Nate Stepping back and thinking about it from a high level with my limited understanding of your application, it seems you are setting animations, then rendering a frame so you see the last frame in the old animations, then you want events to happen for the newly set animations. This is causing issues for various reasons.

Again I believe you're misunderstanding me here. I'm not delaying setting any animations by a frame. That was a suggestion you gave earlier but it wouldn't help our situation here. The main issue is in the update order for AnimationState's Start callback and the SetAnimation not posing the skeleton until the following frame, thus Start is called a frame before the Skeleton is actually posed.

A high level overview of what we are doing is:

Character starts in their Idle state, and the skeleton is posed to their "idle" animation.
A timer fires after X seconds of Idling, setting their state to Punching
Upon entering the Punching state we use SetAnimation("punching")
After the "punch" track's completion callback we set the Character to their PunchReset state.
Upon entering the PunchReset state we use SetAnimation("punch_reset")
After the "punch_reset" track's completion callback we set the Character back to their Idle state.
Upon entering the Idle state we use SetAnimation("idle") and reset the timer to fire again after X seconds.

Here the "punching" animation has an event called "vulnerable_on" which we listen for to set a flag to the game that the character themselves can be punched during this portion of the animation. There is another event in "punch_reset" called "vulnerable_off" to reset this flag.

However we also have a system where when the AnimationState calls its Start callback, we automatically reset the vulnerability flag to "off", this allows us for most animations to not add a "vulnerable_off" event explicitly at the end of many animations.

But in the case above we actually want the vulnerability flag to persist between the "punching" and "punch_reset" animations until "punch_reset" turns it off with the "vulnerability_off" event, so at frame 0 of "punch_reset" we actually call another "vulnerable_on" event to counter the automatic reset. Herein is where the problem lies though, since as you pointed out the AnimationState is only applied once per frame there is a single frame delay between when the AnimationState calls its Start callback and when the "punch_reset" track is actually applied to the AnimationState. This in turn causes a single frame where the vulnerability flag is set to "off" between the automatic reset when the AnimationState calls its Start callback and when the "punch_reset" is applied to the AnimationState (and thus fires its frame 0 events) on the following Update.

I hope that helps to clear up any confusion and misunderstandings, if not please let me know and I can try to expand further and/or provide more information about the issue.

Thanks again for your time and timely responses!

Nate · Aug 30, 2023

ExNull-Tyelor I mean that when a queued animation is set as the current track entry the trackTime is not 0 for that new animation.

Ah, that makes sense. When a queued entry becomes the current entry in update, any leftover time is carried over to the new entry in update (see the AnimationState code).

Thanks for the explanations, it gives a clearer picture of what is happening.

Ideally you have a "model" that is your game state without being tied to the animation system. This takes planning up front and is difficult to shoehorn into an existing project, but it can alleviate these kinds of problems where the game state is dependent on the animation system. There is often some unavoidable dependency when using animation events to change the game state with specific timing, but the less the game state relies on the animation system, the better.

For example, consider if the game state changed as needed regardless of the animation system, then you'd have code that checks if the current animation is the correct one given the current game state. If not, set the right animation. In this way the animations being played are just the visualization for the game state, not the primary driver.

Anyway, that doesn't help you with an existing project, but I mention it because it can be a helpful. Super Spineboy is an example project using such an MVC approach.

IIUC finally, your problems stem from setting new animations in AnimationStateListener events. You update and apply because you want to render the current animations. That triggers AnimationStateListener events which set new animations. Now you have set animations that are not what you actually want to render this game frame.

This setting of the new animations "too early" has a few side effects. The start event occurs, so if you use that to change the vulnerability flag, it won't match the animation that is rendered. It also means you need to avoid calling apply, else you'd lose the last frame from the old animations.

The workaround to check if an animation is being applied for the first time sounds like it can work. Checking trackTime is 0 is only part of it, as you mentioned. The check that update does is current.trackLast - next.delay >= 0, but trackLast is not exposed so you can't mimic the check without modifying the runtime. There is another internal check that is used specifically to determine if an entry has never been applied. That would be simpler to use. We could expose it with a method like:

public boolean wasApplied () {
    return nextTrackLast != -1;
}

You could add that to the runtime and see if it meets your needs.

Another way could be to store the current track entry (for all the tracks you care about), call update, then see if the current track entry changed. If it changed OR if the trackTime was 0 before calling update, then it has not been applied yet. I like wasApplied more though and it seems generally useful.

Another thought is that AnimationState already queues all the events that happen. It needs to do this because it can't execute callbacks in the middle of its work because user code in the callbacks could mutate the AnimationState. When the callback was done and the work resumed, all kinds of things can break. So, the events are queued and only executed at a safe point, often right before a method returns. Search AnimationState for queue.drain(); to see where events are fired.

Anyway, the thought was that you could prevent the queue from being drained (meaning the events fire) when you set new animations in the AnimationStateListener callbacks. The next frame you drain the queue, then the start event happens on the frame you want. This would require some changes to AnimationState, or we could incorporate it into the API, though it feels somewhat complex/low level and less elegant than wasApplied. It's a bit weird to make changes to the AnimationState but not fire the events until later.

You could also queue the events in the same way at the application level, so you can fire them next frame, mentioned earlier. That way you don't actually set new animations until next frame. It's really not the worst solution, as it keeps your AnimationState in sync with what you want to render.

Nate · Aug 30, 2023

Another thought, you could set the vulnerability flag to the current animation when vulnerable. That way you never need to clear it when the animation changes. Check it against the current animation, if it's different, the player is invulnerable.

EExNull-Tyelor · Aug 30, 2023

Nate You could add that to the runtime and see if it meets your needs.

This unfortunately didn't work for the first apply of the queued animations, however by doing a bit of hacking to the runtimes I was able to get something that did:

public bool IsFirstApply()
{
    return trackTime == 0;
}

public bool IsNextFirstApply()
{
    if (next == null) {
        return false;
    }

    return Mathf.Approximately(nextTrackLast - next.delay, 0f);
}

Then in our manual update logic I do:

public virtual void ManualUpdate(float deltaTime)
{
    var baseTrack = Skeleton.AnimationState.GetCurrent(0);
    if (baseTrack != null) {
        if (baseTrack.IsFirstApply()) {
            OnSpineStart(baseTrack);
        } else if (baseTrack.IsNextFirstApply()) {
            OnSpineStart(baseTrack.Next);
        }
    }

    Skeleton.Update(deltaTime);
}

Instead of registering OnSpineStart to the AnimationState's Start callback listener. This solved the issue where the vulnerability flag was marked off for that single frame between when Start was called and when the new AnimationState was applied.

This obviously didn't solve the issue where the Character's State was set a frame before the AnimationState was applied, but I was able to solve that issue as well by just waiting a single frame before actually setting the new State (except for the Player's Character, as we want instant feedback for inputs there). Now the animations are perfectly synced with the Character State, and the OnSpineStart logic is always called just before the new Animations are applied.

Thanks again for your time and all your assistance in helping me solve these issues!

EExNull-Tyelor · Aug 31, 2023

ExNull-Tyelor Again I believe you're misunderstanding me here. I'm not delaying setting any animations by a frame. That was a suggestion you gave earlier but it wouldn't help our situation here. The main issue is in the update order for AnimationState's Start callback and the SetAnimation not posing the skeleton until the following frame, thus Start is called a frame before the Skeleton is actually posed.

Oh, I guess in the end I did need to delay something by a frame: didn't I? It just wasn't the animation I ended up needing to delay by a frame, but the setting of the state lol

Nate · Aug 31, 2023

Woo! I'm glad you got it solved!

This was an interesting case and probably one that others will run into, as few users separate their game state from the animation system. We'll think on how we might be able to improve this.

Can you say why the wasApplied method I provided was insufficient? Queued track entries have nextTrackLast set to -1, so wasApplied would return false. It is only set to another value when the track entry is applied.

EExNull-Tyelor · Aug 31, 2023

Nate Can you say why the wasApplied method I provided was insufficient? Queued track entries have nextTrackLast set to -1, so wasApplied would return false. It is only set to another value when the track entry is applied.

It might've been due to my use-case, but the nextTrackLast wasn't -1 for the frame just before the queued animation was applied for the first time which was causing the queued animations to not call my OnSpineStart method. When I made the "delay, trackTime, trackLast, nextTrackLast" floats public for debugging purposes I printed them out to the console to see what was going on and the nextTrackLast wasn't -1 on the frame the animation should've been applied. In fact, it seemed like the animation was already applied since in one frame I got:
Animation: "1_jug_atk_l" delay: 0, trackTime: 1.233936, trackLast: 1.218325, nextTrackLast: 1.233936
(The last frame for the "1_jug_atk_l")
and the very next frame showed:
Animation: "idle" delay: 0, trackTime: 0.01694712, trackLast: -1, nextTrackLast: 0.01694712
(Which should've been the first frame to apply the "idle" animation.)

I'm not entirely sure why that is, nor what exactly I'm doing to cause the discrepancy either, since the update loop I posted earlier is the only place where we update the AnimationState manually through the Skeleton.Update method. The only thing I can think of is that on the Complete callback listener for "1_jug_atk_l" we set the character's state back to Idle, and upon entering the Idle state we use AddAnimation instead of SetAnimation, though because of the state change I made it doesn't actually enter the Idle state now until the frame after we call SetState. I use AddAnimation there because sometimes we set the Idle state while an animation is still playing (like at the start of a fight) and so I don't want to interrupt the "fight_start" animation with the "idle" animation, but I still want the character to be in the Idle state at that point. Almost guaranteed not the best way to handle that, but it seems to work

Nate This was an interesting case and probably one that others will run into, as few users separate their game state from the animation system. We'll think on how we might be able to improve this.

I think that decoupling our character states from the animation system would've likely been nigh-impossible due to the way that our game works. The spine characters are the game more or less, and I didn't want to have a state for every single animation that we need, especially as we add more characters to the game. Our game is inspired by Nintendo's Punch-Out!! series, and as such there is a Player spine character and one Opponent spine character on screen during the game scene, and each different opponent has their own sets of animations, unique attacks, special attacks, damaged animations, dazed animations, etc. One of our artists isn't very technical either so the names of even similar animations can vary quite a bit from one opponent to another, making code reuse impractical if we wanted each animation to be tied to a specific state. We've also adapted some code logic from our first game in this genre to this sequel, as well as already having made four opponent characters: making that even more difficult without rewriting the entire game logic and changing our animations in these characters. But maybe I just am not good at MVC paradigms and this would've been easier than I thought it would at the start. Originally for our first game the programmer who started the project before me didn't even use a state machine at all and all logic was purely tied to the animations and events, so I at least made that improvement for the sequel

Nate · Aug 31, 2023

OK, thanks for the info. We'll try adding a wasApplied method in 4.2 and make sure it works as intended.

We had an internal discussion about the timing of events. Our official stance is: you don't usually want to make game state changes in AnimationStateListener callbacks. You probably want to defer them until the next frame, using whatever makes the most sense for your game toolkit and programming language.

An alternative to the start callback is to use the new TrackEntry wasApplied method. For other callbacks, like complete, you'll need to defer taking the action some other way.

The same is true for setting new animations, as it is ideal to have the AnimationState stay in sync with the game state. However, you can get by setting new animations in the callbacks, as long as you are careful not to apply the AnimationState until next game frame and you don't make game state decisions based on the AnimationState track entries.

I know fighting games can be very particular, down to a single game frame (I'm a Street Fighter fan, or was until SF5). However, I do think MVC would work just as well for that. The animations are the visualization of the game state, just as they are in every game. In some ways you may have an easier time being precise about the game state timing: your game state is sure to be correct, (almost) no matter what the crazy animators are doing. The crucial part is to come up with a good scheme with a reasonable workflow for building both the game state and animations, and the dependencies when they inevitably interact. That would take some doing.

A related note: most projects would benefit from tools that load a skeleton to test things out, similar to (or even based on) our Skeleton Viewer. That can let animators exercise their work and solve problems before bother the developers. Such tools can also and define game specific configuration, which can also reduce the load on developers by having non-technical people work on config. Strangely many projects skip making app specific tools, it's a bit unfortunate.

EExNull-Tyelor · Aug 31, 2023

Nate OK, thanks for the info. We'll try adding a wasApplied method in 4.2 and make sure it works as intended.

Awesome! That would mean I don't need the hacky methods I added to TrackEntry

Nate We had an internal discussion about the timing of events. Our official stance is: you don't usually want to make game state changes in AnimationStateListener callbacks. You probably want to defer them until the next frame, using whatever makes the most sense for your game toolkit and programming language.

An alternative to the start callback is to use the new TrackEntry wasApplied method. For other callbacks, like complete, you'll need to defer taking the action some other way.

The same is true for setting new animations, as it is ideal to have the AnimationState stay in sync with the game state. However, you can get by setting new animations in the callbacks, as long as you are careful not to apply the AnimationState until next game frame and you don't make game state decisions based on the AnimationState track entries.

To be completely honest with you, I have no idea how I would program these systems in a way where the animations are completely decoupled from the character states. We rely on a lot of TrackEntry.complete listeners, as well as a lot of custom event listeners for firing off game logic, e.g. knowing when a punch "connects" and knowing when a character hits the ground, as well as knowing when to turn on/off those vulnerability flags I mentioned before.

I'm also not sure how else we would know when to change states from say Punching back to the Idle state, since the duration of idles and punches vary from character to character, even oftentimes between individual attacks for these characters. I suppose we could just have timers and use the duration of these TrackEntrys to set the timer duration, but then that doesn't feel very decoupled from the animation system... I'm probably just not good enough as a software engineer to see the clever solution here however.

Nate A related note: most projects would benefit from tools that load a skeleton to test things out, similar to (or even based on) our Skeleton Viewer. That can let animators exercise their work and solve problems before bother the developers. Such tools can also and define game specific configuration, which can also reduce the load on developers by having non-technical people work on config. Strangely many projects skip making app specific tools, it's a bit unfortunate.

If most projects are done like ours is, then likely the reason is due to a lack of time and hands to implement those tools . I'm the sole engineer for our new game projects right now, and we have one, sometimes two artists working on the artwork at any point in time for our games. We are pretty limited on the resources we can devote to our games, and as such we don't really have the time to develop custom tools for our projects and mostly have to make due with what we're given or can find for relatively cheap: especially since projects like these always take more time than we initially hoped for. Anything outside the scope of our projects, like custom app specific tools, would likely have to be made by me on my own personal time. I reckon small teams and lack of resources, like our team has, is the biggest reason why app specific tools often get neglected, even if they might marginally speed up the future workflow...

Nate · Aug 31, 2023

ExNull-Tyelor I suppose we could just have timers and use the duration of these TrackEntrys to set the timer duration, but then that doesn't feel very decoupled from the animation system.

Probably something like this makes sense. There can still be benefits to decoupling, even if it breaks down a little here and there. This is all just conjecture from my side anyway, you are the one who has actually implemented the app!

ExNull-Tyelor If most projects are done like ours is, then likely the reason is due to a lack of time and hands to implement those tools

True, but it could be a net gain. People can take the Skeleton Viewer source and hack on features, or make a trimmed down version of their game that allows skeletons to be swapped out, loaded from a file when the file changes. Once you do a tool or two like this, it shouldn't take a huge amount of time. Still I am familiar with never having enough time.

EExNull-Tyelor · Aug 31, 2023

Nate Probably something like this makes sense. There can still be benefits to decoupling, even if it breaks down a little here and there. This is all just conjecture from my side anyway, you are the one who has actually implemented the app!

Yeah that would get messy though when trying to change the state due to player actions though, like countering their attack during the vulnerable portion of the opponent character's attack to send them into a Dazed State mid-animation though. I definitely appreciate your conjecture too! It's nice to get other people's opinions on implementation when I'm the sole engineer for the game right now For now we'll likely be using the frame delay system for the state changes unless (until) we find that it breaks something else horribly, but it seems to work good enough for now lol

Nate True, but it could be a net gain. People can take the Skeleton Viewer source and hack on features, or make a trimmed down version of their game that allows skeletons to be swapped out, loaded from a file when the file changes. Once you do a tool or two like this, it shouldn't take a huge amount of time.

I could see something like that being useful for our artists, though it'd be tricky to get right for our use case. It'd mostly be used for seeing animation transitions and being able to queue the animations and tracks to see how they work together. This way they could find out how much "popping" from one animation to another they can get away with before it looks bad. But outside of that I'm not sure it'd be of much use: and the Spine Editor seems to do that well enough for them, I think.

Nate Still I am familiar with never having enough time.

I think this is the bane for all us software engineers, isn't it So much so that "good enough" is basically our entire job now