Hello,
I followed the YouTube tutorial below and was able to complete the entire workflow successfully using a Character Creator character.
https://youtu.be/nyPNP-S92QI?si=-py_vMBtowfxVuUZ
I then created my own custom character and followed the tutorial step by step in Unreal Engine. However, I am encountering several issues while trying to apply the same workflow.
Issue 1: Lip Sync Is Not Working
When the character is speaking, the mouth does not open or move at all.
The body animation correctly switches to the "CC5__Anim_Stand_Talk"** state, but there is no facial movement during speech.
I also changed all available lip-sync settings in Unreal Engine to ARKit Blendshapes, but the result remained the same.
I suspected that the issue might be related to bone names, blend shape names, or other settings used throughout the pipeline, so I attached screenshots showing the relevant configurations.
In addition, since it seemed that the mouth movement was not driven entirely by blend shapes, I added weights to the Jaw Root Bone.
As expected, both the blend shapes and jaw bone movement appeared to influence the lip sync, and after adding jaw weights, the character’s mouth showed slightly more movement while speaking.
However, when I compare my character to the Character Creator character used in the tutorial, it seems that blend shapes contribute much more significantly to the facial animation.
For example, the Character Creator character shows visible facial movements such as eyelid and eyebrow motion while speaking, which appear to be driven by blend shapes.
I would like my custom character’s blend shapes to function in the same way.
Issue 2: Character Shifts Position When Switching Between Idle and Talk Animations
During simulation in Unreal Engine, the character remains in an Idle state while listening and switches to a Talk animation while speaking.
However, when the animation changes from Idle to Talk, the character slides approximately one step to the right on the screen (which corresponds to the character’s left side).
When speaking ends and the animation returns to Idle, the character slides back in the opposite direction.
As a result, the character visibly shifts position every time it starts or stops speaking.
Issue 3: Body Parts Intersect During Animation
When using the default Character Creator animations, I encounter what appears to be a body proportion mismatch.
For example:
-
The arms sometimes pass through the chest or stomach.
-
The legs sometimes intersect with each other.
The animations themselves play correctly, but they do not seem to fit my character’s proportions very well.
What I Tried for Issues 2 and 3
To solve the animation-related issues, I attempted to edit the animations directly in Unreal Engine.
-
Opened the Animation Sequence in a Level Sequence.
-
Baked it to a Control Rig.
-
Manually adjusted the animation.
For Issue 2:
- I slightly adjusted the character’s position to compensate for the unwanted movement.
For Issue 3:
- I modified the arm and leg motions to reduce body intersections.
After making the adjustments:
-
I baked the result into a new Animation Sequence.
-
Saved it as a new animation (for example, “talk_re”).
-
Replaced the original Talk animation in Convai_Reallusion_AnimBP with the new animation.
-
Compiled and saved everything.
-
Tested it again in simulation.
However, the result did not change at all.
It almost seems as if the modified animation sequence is not actually being used.
Additional Investigation
While researching solutions for the body intersection issue, I found recommendations suggesting that animation adjustments should be made through a Control Rig asset.
While tracing the animation nodes and Blueprint connections, I also discovered that several Control Rig assets already exist in the project.
However, I am not sure which Control Rig should be modified or what exactly needs to be adjusted.
Could you please let me know the correct way to address these issues?
If there is any additional information, file, or screenshot that would help diagnose the problem, I would be happy to provide it.
Thank you very much for your time and assistance.
