Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement SkeletonRetarget and overhaul some animation features #56902

Conversation

TokageItLab
Copy link
Member

@TokageItLab TokageItLab commented Jan 18, 2022

Implement SkeletonRetarget based on reduz's proposal (godotengine/godot-proposals#3379).

This PR includes follows:

This requires #55840 because global_pose had a serious problem of not being able to get IK and other skeleton modifire transforms. We also needed #54205 to test the AnimationTree.

#55218 and #49411 are bug fixes that are not directly related to retarget, but I believe they need to be fixed in the future, and they cause conflicts with this PR, so I included them in this PR to resolve the conflicts in advance.

Fixed #43848. Fixed #48526. Fixed #49431. Fixed #54407. Fixed #55838.


Implement SkeletonRetarget

01

SkeletonRetarget (Node)

Transfers the transform between the two skeletons. I implemented three types of resources for retarget. It corresponds to the "Retargeter" resource that reduz mentioned in the proposal, which is separated into several parts to make it easier to reuse.

  • RetargetProfile / RetargetRichProfile
  • RetargetBoneMap
  • RetargetBoneOption

02

RetargetProfile provides the intermediate bones. The source skeleton and target skeleton bones are mapped to each other through intermediate bones using RetargetBoneMap. The transfer value is calculated differently by RetargetMode. The default mode is RetargetMode::RETARGET_MODE_GLOBAL. You can set it for each intermediate bone using RetargetBoneOption.

RetargetProfile (Resource)

This resource is special since it is only used in the editor. It does not have any effect on the retargeting calculation.

The RetargetBoneMap is needed for each skeleton with a different bone name, but the keys must be the same. Therefore, to make it easier to generate the same keys in those, RetargetProfile lists the keys as intermediate bones.

You need a RetargetProfile when you register a key to RetargetBoneMap or RetargetBoneOption in the editor. However, once you register the keys in them, you can discard the RetargetProfile if you don't need it. Also, you can register keys in them with the script always.

03

RetargetRichProfile defines textures, coordinates, and groups in addition to the intermediate bones. It provides a GUI with the buttons placed on the silhouette image to register the keys to RetargetBoneMap and RetargetBoneOption.

04

If users have defined their own RetargetProfile, it is recommended that they also include this profile when sharing retarget animations to make convert easy between users.

I'm still undecided on how to implement the presets that Godot has. Should I just distribute gd_humanoid.tres in the Asset library, or should I provide a hard-coded uneditable GDHumanoidProfile class that inherits from RetargetRichProfile? A similar implementation used to exist, default_env.tres, but that has been replaced by an internal implementation.

RetargetBoneMap (Resource)

A map of skeleton bone names with intermediate bone names as key. To register a key in the editor, a RetargetProfile is required. In the inspector, maps are organized by RetargetProfile, but if there is no RetargetProfile, they are listed in the unprofiled bones section.

RetargetBoneOption (Resource)

Set the options for each intermediate bone. It has RetargetMode as one of the options. The properties that should be set for each intermediate bone and do not need to be set for each skeleton, will be put together in RetargetBoneOption instead of RetargetBoneMap.

RetargetMode::RETARGET_MODE_GLOBAL

Transfers the global transforms in the model space relative to the bone rest.

If the models are in the same stance, the animations will basically transfer well. However, if the models are in different stances, such as T-stance and A-stance, the transfer will be made looks strange.

2

However, the retargeter should not involved in that. Potentially, it make sense that an add-on such as T-poser makes match the stance between them.

BTW, the bones do not allow for shearing, so it will be orthogonalized implicitly if the animation includes scaling.

RetargetMode::RETARGET_MODE_LOCAL

Transfers the local transforms relative to the bone rest (same as the default animation format in Godot 3).

1.mp4

For example, for bones that are slanted even at T-stance, such as fingers, the joints may rotate in odd directions. In such a case, it is recommended to use RETARGET_MODE_LOCAL with a restriction in the bone rest axis such as "+X axis rotation to bend".

RetargetMode::RETARGET_MODE_ABSOLUTE

Transfers the local transforms relative to the initial value of Transform (same as the default animation format in Godot 4).

If the bone rest does not match completely, it will look bad. Possibly, it could be useful for transferring animations between different stances, such as T-stance and A-stance, if the Rest rules match, but it is basically not recommended to use it for transferring the transforms between different models.


AnimationPlayer

SkeletonRetarget transfers the transforms from the source skeleton to the target skeleton, but this means there are always two skeleton in the scene, which can make for worse performance in some cases.

Therefore, I implemented the properties for retarget in the AnimationPlayer to make it easier to manage the animation and replace the target model.

05

Also, as a variation of Pos/Rot/Scl3DTrack, implemented retarget track which has RetargetMode (actually it is just Pos/Rot/Scl3DTrack which has special path like :Hips, only subpath).

06

To extract the retarget tracks, it need a source skeleton and a RetargetBoneMap. In other words, once the retarget tracks are extracted, the source skeleton is no longer needed.

You can create the retarget tracks from the options.

07

The retarget track will transform to the skeleton assigned to the AnimationPlayer.

3.mp4

This will make it easier to manage animation resources and distribute retargeted animation packages grouped by RetargetProfile in asset libraries, etc.

BTW, I'm worry about that should it allow multiple retarget skeletons to be assigned to an AnimationPlayer, in case one animation contains multiple skeletons? In my opinion, that can be solved by splitting the animation into multiple animations and using AnimationPlaybackTrack.

Note:
For rotation, the global retarget track will calculate the multiplication Quaternion 3 extra times per bone, and the local retarget track will calculate the multiplication 1 extra time. As a reference, for a single model with 54 bones, using or not using the global retarget track made a difference of about 0.2ms in the process.

I think this is not enough of a performance hit to be a problem for several models, but I would recommend matching the axes in the DCC software and using local retarget if possible (the delay will be 1/3). Of course, it goes without saying that if you have hundreds of enemy models, you shouldn't use global/local retarget in runtime in the first place; You can restore retarget tracks into them.


TODO in the future

To adjust position value between the models have different height

The position values are transferred regardless of the model height. For example, transferring the root motion from a short model to a tall model will cause a slip. Position values are may used in weapon throwing animations. For toe and knee stands, we may need to specify specific bones in the animation. For example, a toe stand requires the length of the Foot-Toes. Kneeling requires a length of LowerLeg-Foot. And its length varies from model to model, both require a Hips bone to move.

For now, I think the only problem is the root motion, but I don't know if I should solve it in the retarget process. It may be solved by implementing an AnimationTree with a root_scale property. Or, maybe we need to store RootDistance (ModelScale / HipsHeight) as a single float value in RetargetBoneMap in the future. So, I make it is out of the scope of this PR.

For compressed tracks

I'm a little concerned about the conversion of compressed tracks, since it convert all keys when converting to Retarget tracks, but for compressed tracks it may need to do the same for all byte arrays. This will be worked out after the implementations around of the compressed/stream track is finalized.

Retarget track renamer

We may want a utility to match the names between extracted animations with different profiles, it can be added later.

Adjusting the transform value when the target model has fewer bones than the source model

For example, if you are transferring motion from a model with four spine bones to a model with three spine bones, you will need to add the deformations of the removed bones to one of the three in order for the looks to match.

I experimented with this a bit, but aborted because it seems that we would need to change the cache format of the animation system to support this.

One solution that can be implemented quickly is to add a virtual bone of zero length to the parent or child. This helper method can be added later.

skeleton_retarget_sample_project.zip

@TokageItLab TokageItLab requested review from a team as code owners January 18, 2022 13:53
@TokageItLab TokageItLab changed the title Implement skeleton retarget and overhaul some animation features Implement SkeletonRetarget and overhaul some animation features Jan 18, 2022
@Calinou Calinou added this to the 4.0 milestone Jan 18, 2022
@Calinou Calinou added the bug label Jan 18, 2022
@Calinou
Copy link
Member

Calinou commented Jan 18, 2022

Is retargeting limited to Skeleton3D, or does it also work for Skeleton2D? The class reference should make this clear.

@TokageItLab
Copy link
Member Author

TokageItLab commented Jan 18, 2022

@Calinou It's 3D only, do I need to rename the class to SkeletonRetarget3D in addition to putting the note in the SkeletonRetarget document (class reference)?

@Calinou
Copy link
Member

Calinou commented Jan 18, 2022

@Calinou It's 3D only, do I need to rename the class to SkeletonRetarget3D in addition to putting the note in the SkeletonRetarget document (class reference)?

I'm not sure if the class should be renamed – I'll let others give their opinion on this 🙂

In Godot, 3D-only classes such as VoxelGI, ReflectionProbe and Decal generally don't have a 3D suffix.

@TokageItLab TokageItLab force-pushed the skeleton-retarget-and-animation-overhaul branch from c256b84 to cd68419 Compare January 18, 2022 16:35
@TokageItLab
Copy link
Member Author

TokageItLab commented Jan 18, 2022

Updated the document for now, clarifying that "SkeletonRetarget is for Skeleton3D" in the class reference.

I don't have much knowledge about Skeleton2D. If it has the concept of rest/pose, then I think a similar implementation of SkeletonRetarget is possible. However, it may not possible to implement a retargeting track for Skeleton2D in AnimationPlayer. The reason is that Skeleton2D does not have its own track type for transform.

In Skeleton3D, only Pos/Ros/Scl3DTracks are allowed for bone transform. But, Skeleton2D bone transforms are shared with normal properties, and there are two types of tracks: Bezier Tracks and Property Tracks. This makes retarget tracks for Skeleton2D difficult to implement.

Although, I think 2D assets don't have much concept of stance, unlike 3D assets, so I guess there is not much demand for implement them for Skeleton2D. In other words, there are few models that have the "same stance" and different bone naming conventions and rest as an asset; It's means, there are few animation packages. So I guess it is rare case that need to retarget for 2D, since multiple models in one product may be created by the same person.

I also wait for others' opinions.

@TokageItLab TokageItLab force-pushed the skeleton-retarget-and-animation-overhaul branch from a9af0da to a72ecb3 Compare February 28, 2022 09:20
@TokageItLab
Copy link
Member Author

TokageItLab commented Mar 5, 2022

We're struggling with @reduz on whether to record retarget_mode in Animation::Track or Skeleton3D::Bone.

The fundamental difference between my approach and reduz's is the point that "animation requires or does not require a Skeleton from which the animation is generated".


The premise is that animation data can have information about the original Skeleton through pre-calculation. However, this is destructive and the information of the original Skeleton cannot be retrieved from the animation.

For example, let's say the bone rest value is 1 and the animation value is 3.

3 - 1 = 2

In this case, a value of 2 is required for retargeting. Therefore, the animation only needs a value of 2 to be ready for retargeting.

What is being debated is where to record the "information as to whether the value is a pre-calculated value or not".

I believe it should be stored in the same place as the value. In other words, retarget_mode should be recorded in Animation::Track. I also believe that the original Skeleton is no longer needed at this point.

Reduz thinks it should be selected as a preset when importing the model and store the retarget_mode in Skeleton3D::Bone. And, he thinks that the animation should be pre-calculated for performance, but the original skeleton should keep to be stored somewhere.


I consider the reduz's method to be problematic in two main points.

1) If a preset is changed after imported animations, all animation values must be recalculated

Also, the original skeleton which is source of imported animations is required for recalculation.

It means that they will be broken if the original skeleton has been removed and does not exist, or if there are duplicates or newly created animation that are not tied to the original skeleton.

Untitled-1

2) It is to be impossible to blend LocalAnimation and GlobalAnimation

This is clear if we assume that presets are to be used, but if you store the retarget_mode in Skeleton3D::Bone without using a preset, they will not blend well.

LocalAnimation and GlobalAnimation cannot be blended directly because they are recorded in different coordinate spaces by pre-calculation.

AbsoluteAnimation (glTFAnimation, Godot4 Default)

  • Record
AnimQ = SrcPoseQ
  • Playback
TgtPoseQ = AnimQ

LocalAnimation (Blender, Godot3 Default) (a.k.a RestAnimation)

  • Record
AnimQ = SrcRestQ.inv * SrcPoseQ
  • Playback
TgtPoseQ = TgtRestQ * AnimQ

GlobalAnimation

  • Record
AnimQ = SrcParentGlobalRestQ * SrcPoseQ * SrcRestQ.inv * SrcParentGlobalRestQ.inv
  • Playback
TgtPoseQ = TgtParentGlobalRest.inv * AnimQ * TgtParentGlobalRest * TgtRest

In order to blend them, their values must be restored to the target skeleton bone's coordinate space, but to do so, it is necessary to know whether the animation was "pre-calculated as LocalAnimation" or "pre-calculated as GlobalAnimation".

broken_way

There is no need to think too hard, it is obvious that if you have an animation extracted as GlobalAnimation, its playback will be broken if LocalAnimation mode is applied to the target skeleton bone.

We are discussing hard about this case of blending LocalAnimation and GlobalAnimation because reduz claims that it is a very niche case.

For blending LocalAnimation and GlobalAnimation:

For example, I have Model A and Model B, with rest optimized for my project. These models have different rest values, but the rules for the joints match, such as +X rotation bends, and parent to child is Y-axis and roll.

The basic animation is created by Model A and shared with Model B as LocalAnimation.

It is important to understand that LocalAnimation is less likely to cause bone breaks if the models match the joint rules. This is very useful in cases where different models have different orientations, such as the thumb. If you have struggled with finger control in Unity models, this is probably easy to understand.

However, I want to add more animations, so I want to import animations from Model M, where Rest rules do not match.

If retarget_mode is recorded in Animation::Track:

the animation of Model M can be imported as GlobalAnimation and applied to Model A or Model B, moreover it can be blended with LocalAnimation.

If retarget_mode is recorded in Skeleton3D::Bone:

there are two ways.

Import the animation of model M as GlobalAnimation. However, re-import the animation of model A and model B as GlobalAnimation instead of LocalAnimation. At this time, the advantage of LocalAnimation's bone breaking resistance is lost, and the playback animation of the fingers will be dirty. This means that if there is even one model in the project with a different rest rule, only GlobalAnimation can be used.

Alternatively, the rest rule of Model M could be matched to Model A and Model B by somehow, but the process of validating rest rule is complicated. Also, if you do simple re-rests, it produce messier animations.


TL;DR

  • I believe that having retarget_mode in Animation::Track would make it easier to share only animations among users and projects
    • Reduz believes that .tres(animation) should not be allowed to be used in separate from Skeleton in the first place, also he said that bone animations should not be generated or edited on Godot in the first place
      • Unity and Unreal have specific animation files for retargeting. If those files cannot be managed independently from skeleton, the project will increase in complexity
  • I use LocalAnimation because I need to care about finger animation in certain animations. At the same time, I use GlobalAnimation for other animations where I don't need to care about finger animation. Then I blend them together
    • Reduz says that "this case is that you are the only one who will use it"
      • I think that's obvious because Godot can't do it at the moment, and I suspect there are others who would like to do that workflow

We need others' opinions as it is difficult for us to resolve the discussion on our end.

Please give us your opinion if possible: @fire @lyuma @SaracenOne @AndreaCatania @TwistedTwigleg @nonunknown

If any other developers are interested in character animation or retargeting, please join the discussion.

@Zireael07
Copy link
Contributor

Hm. Looking at my character models, who have different bone names and rest poses despite coming from the same author and pack, I think I would mostly use GLOBAL mode. However, for some things, e.g bending arms to wave, or legs to sit down, I think LOCAL would be more intuitive (just bend this by 90 degrees, this other by 90 degrees, done).

So yes, +1 to something that allows mixing GLOBAL and LOCAL.

@TokageItLab TokageItLab force-pushed the skeleton-retarget-and-animation-overhaul branch 4 times, most recently from 984509f to 974e153 Compare March 7, 2022 14:43
@TokageItLab TokageItLab requested a review from a team as a code owner March 7, 2022 14:43
@TokageItLab TokageItLab force-pushed the skeleton-retarget-and-animation-overhaul branch 3 times, most recently from 44b64c0 to 13fe177 Compare March 10, 2022 23:45
@lyuma
Copy link
Contributor

lyuma commented Mar 14, 2022

In short, having played with this, and talked this over with others, I'm trying to boil down some of the things I observed.

First of all, Local (rest) and Absolute modes are not good enough for most of the common retargeting cases. As a test,. I transferred a local animation from one character to an Xbot model, and the animation was broken. See here: left is Global (GLO) which works well. Right is Local (LOC) which has arms and body backwards (image below).

So the question is: when is Global not good enough? Well, it is not well suited for finger bones. By its very nature, it does not know how finger bones work, since it is working on the "silhouette" of the model.

This means none of these retargeting modes are good enough on their own. So what can we do about it?

  • Perhaps it is possible to create an importer which twists fingers in the Bone Rest matrix into a configuration suitable for Global retargeting? I can't say whether or not this is possible, but if this could be solved, then maybe we can use Global mode for everything?
  • It could be declared that finger bones shall be local, and all other bones shall be global, and everyone making assets should agree on finger rotations. This might not always be possible, but this is my interpretation of reduz's idea of storing the PlaybackMode in Node B in Tokage's diagram.
  • Or it could be left up to each animation artist / importer which to use, and then you're going to have a mixture of modes. If so, I see two ways to deal with it:
  1. The animations are stored in global along with skeleton and converted using the source skeleton. This sounds ok in principle if some conversion tool could be made, but I would be afraid to deal with converting animations on the fly at runtime.
  2. the animations must point to the RetargetOption they were generated with (or a per-track retarget mode), so that these differently sourced animations can be mixed.

from what I can understand, "2" is what Tokage is proposing. I don't see many other solutions to this. Fundamentally, if it is the case that animations made from different models are needed to be mixed, I don't see a viable solution other than the last one.
Global vs Local comparison .png

@TokageItLab
Copy link
Member Author

TokageItLab commented Mar 14, 2022

@lyuma

First of all, Local (rest) and Absolute modes are not good enough for most of the common retargeting cases.

Yes, it has been proven in Godot 3.x (which uses LocalAnimation as default) that it is not sufficient for retargeting.

For the time being, GlobalAnimation should be the default for retargeting to avoid confusion for light users. However, for advance users, for parts with delicate joints such as fingers, it is more appropriate to use LocalAnimation (RestAnimation) with matching RestRule.

MMD (MikuMikuDance), a dance creation software, basically only performs GlobalAnimation, but there are plug-ins that adjust fingers' Rest for editing, which proves that there is a demand for this method.

For about AbsoluteAnimation, the only use case I can think of for transferring AbsoluteAnimation to another model is when you want to animate a bone which is child of the root of the IK joint as an IK target, regardless of the model's figure. It does not seem to be completely useless.


Perhaps it is possible to create an importer which twists fingers in the Bone Rest matrix into a configuration suitable for Global retargeting? I can't say whether or not this is possible, but if this could be solved, then maybe we can use Global mode for everything?

This is in principle the same as making an A-pose into a T-pose; doing it automatically is not sufficient. There must always be manual confirmation, or the joint may bend in a strange direction.

For reference, some MMD plug-ins will align the RestRule to some degree, but correct looks requires manual adjustment finally.

However, even if the rest rule matches, GlobalAnimation should be used for Hips and Spine.

The reason is that they do not require as much rigorous deformation because the joints are not clearly visible, and because they are closer to the root, there are more errors that depend on the model.

For example, when sharing "an animation of a horizontal waist twist created with a model that does not have an S-shaped waist", using LocalAnimation may cause the arms to turn at an angle on a model with an S-shaped waist.

In the same way, Neck-Head and Shoulders should basically use GlobalAnimation, but in some cases such angled rotation may be desired, so blending with LocalAnimation should be possible.


So the question is: when is Global not good enough? Well, it is not well suited for finger bones. By its very nature, it does not know how finger bones work, since it is working on the "silhouette" of the model.

It could be declared that finger bones shall be local, and all other bones shall be global, and everyone making assets should agree on finger rotations. This might not always be possible, but this is my interpretation of reduz's idea of storing the PlaybackMode in Node B in Tokage's diagram.

Here is a more common case.

As an assumption, the quality of retargeting for fingers and limbs can be ordered as follows:

[Highest Quality]

  1. Local animation(rest rule matched manually)
  2. Global animation
  3. Local animation(rest matrix matched automatic)
  4. Absolute animation(rest matrix matched automatic)
  5. Absolute animation(rest rule matched manually)
  6. Local animation(rest rule doesn't matched)
  7. Absolute animation(rest rule doesn't matched)

[Lowest Quality]

There is a game which player collect character units.

The idle animations of the characters are created and shared using the Local animation(rest rule matched manually), which has the highest quality, since they are rendered large on the home screen.

During missions, characters are rendered smaller, but instead require many animations. Therefore, animations are imported from various assets and shared using Global animation.

However, idle animations are also used during the mission, so Local animation and Global animation must be blended.

If we adopt redus's method, we need to convert some animations; If the idle animation is Global animation, the retarget quality goes down. Also, to make Local animation(rest rule matched manually) for the mission animations would require manual conversion of many assets. To reduce the effort, we can convert to Local animation(rest matrix matched automatic), but it will be of lower quality than Global animation.

Blending Local animation with Global animation will give us good retargeting results at a lower cost, thus maximizing cost effectiveness.


Or it could be left up to each animation artist / importer which to use, and then you're going to have a mixture of modes. If so, I see two ways to deal with it:

  1. The animations are stored in global along with skeleton and converted using the source skeleton. This sounds ok in principle if some conversion tool could be made, but I would be afraid to deal with converting animations on the fly at runtime.
  2. the animations must point to the RetargetOption they were generated with (or a per-track retarget mode), so that these differently sourced animations can be mixed.

from what I can understand, "2" is what Tokage is proposing. I don't see many other solutions to this. Fundamentally, if it is the case that animations made from different models are needed to be mixed, I don't see a viable solution other than the last one.

Yes, I believe that LocalAnimation and GlobalAnimation need to be blended. And to do so, it is necessary to mark the PlaybackMode into AnimationTrack with pre-multiplying (solution 2), as shown in the diagram above.

Alternatively, there are solutions that store the original skeleton in the AnimationTrack without pre-multiplying (solution 1), but this implementation should be avoided for performance and redundancy reasons.

Regardless of which solution is used, marking is required on the AnimationTrack side for blending GlobalAnimation and LocalAnimation, not on the Skeleton::Bone side.

Also, in order to absorb height differences in the root motion, adjustments using the hips height of the model will be necessary, and I think that this information needs to be included in the AnimationTrack as well.

I don't know if it should be stored directly as float hips_height or normalized as postion track value / hips_height, but it should be adjusted when there is a rest/global marking.

@TokageItLab TokageItLab force-pushed the skeleton-retarget-and-animation-overhaul branch from 13fe177 to 5fcca45 Compare March 17, 2022 20:11
@TokageItLab
Copy link
Member Author

TokageItLab commented Apr 15, 2022

I wanted to keep RetargetMode in the AnimationLibrary implemented in #59980 and be able to blend different RetargetModes, but @reduz rejected that.


Part of the discussion:

Tokage:

Let me explain the use case again.

In an extreme use case, a normal human and a robot whose knee Rest is facing 180 degrees in the opposite direction, want to share an animation of bending their knees. In this case, the human and robot have the same Rest-Rule, so RestAnimation can bend the knee in the correct direction. However, with GlobalAnimation, the knee will bend in the wrong direction.

A 180-degree error is fatal. In fact, if you have several different human models, the finger angles can be off by as much as 15 degrees from one model to the next. This is because fingers do not have a proper format like T-pose. Therefore, it is necessary to use RestAnimation for fingers and to match Rest-Rule as much as possible.

The problem is that when the Rest-Rules share animations of different models, the animations are broken unless they are GlobalAnimation.

Or the Rest-Rules must be matched. This should be done for everything, but this is very costly. Your proposal requires this.

Therefore, the best way to maximize performance is to use RestAnimation for animations like dancing, which requires a large finger on the screen, and use GlobalAnimation from different assets for situations like moving a field, which requires many animations. I believe that this is right way.
If the number of models is 10 and the number of animations is 50, my proposal only requires 10 Rest-Rules to be matched, but your proposal requires it for 60
Incidentally, the Rest-Rule matching must be manually done. Assistance by script is possible, but visual confirmation is always required

reduz:

I understand as much, but this sounds like its something you will align per model, not per animation

Tokage:

Well, in reality, since 50 animations are not all on different models, the number of animation source models will be about 10, but it is still a hassle to make 20 adjustments

reduz:

I understand where you are coming from, but I insist again that this is now the kind of situation you will find when doing actual game development.
It sounds to me that this is something that may commonly happen when you share MMD animations, and it makes sense, but here as a developer you have more control of what you are getting into your game.

Tokage:

For example, if we are going to allow users to upload custom animations, such as VRchat, we will be forcing them to follow the only one Rest rules

reduz:

I understand this, but I feel this is not what we are aiming for in Godot, this is a very specific use case for one specific type of game. I think you will have more luck here writing your own animation format and blending logic for a game like this
You have to think that there are a zillion types of games made with Godot, if we would make Godot work exactly how every single of them needs, Godot would be full of features that nobody would use.

Tokage:

So you are saying that light users don't use RestAnimation and advanced users who use RestAnimation should suffer from Rest matching on Godot

reduz:

I am not sure if I would consider those advanced users, its a very specific type of game that needs this. To begin with, games with some significant (not even high) budget just create their own animations.
So for the most part, its a very very specific type of game that needs something like this.

Tokage:

Indie game developers also create their own animations, but it is impossible to create all the animations, so we need to blend the asset animations with your own animations

reduz:

I do agree with this, but from there to trying to implement a very complex retargeting system that only you need so far there is a long distance.


Also, this PR is starting to get old, so I am abandoning it for now. I think the GUI for mapping and the functions to extract transforms for retargeting are salvageable for future implementations of Retarget.

I think more user voices are needed on this. Unfortunately, I feel that not most people who use Godot are familiar with the 3D character workflow since Godot is mainly used for 2D Game. And Godot's Animation team alone cannot make a fair enough decision.

If anyone interested in 3D character workflow and Retarget read the log above, it would help us a bit if you could leave a comment. Or we would like to have a specific discussion on Contributors Chat.

@Zireael07
Copy link
Contributor

As a prospective user (I have several character assets, one of them is not from the packs the others are, and even the others don't share skeletons even though they share the author) I would like even a basic retargeting system to make it in. More advanced stuff like what you seem to be discussing with reduz above can go in in a further PR IF it turns out to be needed.

@JoanPotatoes2021
Copy link

JoanPotatoes2021 commented Apr 17, 2022

Here's how I would expect animation retargetting to work, my opinion in a grain of salt, as an artist and game developer I can work on my own animations for anything I need, however what I can't do is to customize/mass produce animations to each different character I want with different proportions, that would take too much work, unreal show a use-case with it's animation retargeting that describes exactly what I want as a developer, to re-use animations on a somewhat similar or identical humanoid skeleton with different proportions. Here's the link to unreal doc:

https://docs.unrealengine.com/4.26/en-US/AnimatingObjects/SkeletalMeshAnimation/AnimationRetargeting/

My idea of skeleton retargeting is not to be able to gain animation assets from different skeletons from some rig on the internet, but to increase the possibilities with my animations to adopt new proportions. Here are some photos of what I described gathered from the unreal docs.

55

3

2

And here's a definition from unity's "Retargeting of Humanoid animations" section:

66

I could only dream of a animationblendtree node filtered on which animation I want to be retargetted on which bones.

Edit: So having re-read some the posts above, if I understood it correctly, my use-case would lean on global animations, doing local animations is great because it opens a whole world to do procedual animations, however I also believe it's also going to turn into a can of worms and introduce too much complexity because you need to account for unwanted rotations, alignment issues, bone twisting, etc. I would not want to edit skeleton animations inside godot, for that I would use another package to work on, like blender, I mainly want to direct animations in godot, you go there and you start from here, at the maximum on modifying animations, experimenting with animation blendings with different segmented animations in the blendtree is what I would want from the engine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment