Hello ,
I’m currently working with Unreal Engine 5.5.4 and have created a MetaHuman character. I’ve animated the body using custom keyframe animation in the Level Sequencer, not using Live Link or mocap. I’m also using the Convai plugin for AI chatbot integration, which is working correctly in terms of generating audio responses.
My goal:
Keep the custom body animation exactly as it is
Integrate real-time AI-based facial lip sync (only override the face with lip movement based on chatbot audio)
The problem:
I’m unable to integrate Convai’s AI audio lip sync with the MetaHuman’s face while keeping the body animation unchanged.
I’ve already watched several tutorials and gone through documentation, including:
- UE5 MetaHuman Audio to Face | Auto Lip Sync Tutorial
- Convai AI Character Integration Tutorial
- UE5 Live Facial Capture with Convai
…but still, it’s not working as expected. The lip sync either doesn’t trigger, or it conflicts with the custom animation.
I’ve also attached a short demo video to show my current setup and what I’m trying to achieve.
I need help with:
- How to blend facial AI lip sync (Convai audio) with a pre-animated body?
- What is the correct way to override only facial animation?
- If anyone has done something similar, can you guide me on what node setup or plugin configuration might help?
Thanks in advance to anyone who can help guide me here!