How to update Convai plugin to fix MetaHuman lip sync issues

Integrations Unreal Engine Plugin
1

What do you need help with?

Question

Installation method

FAB

Issue area

Performance / Latency

Impact

Normal - something is not working as expected

Unreal Engine version

5.7.4

Convai Unreal Engine Plugin version

4.0.0-Beta 15

Project setup

Blueprint

Target platform

Unreal Editor

Where does it happen?

Both sample and custom project

Short summary

Exaggerated viseme lip-sync, not completely synced, Metahuman starts Idle anim whilst talking.

What happened?

Building a close-up “talking head” demo on a fixed CineCamera — MetaHuman, Convai v4.0.0-beta.15, UE 5.7.4, on a Vagon cloud Windows workstation.

Already resolved via documented config: Our .ini had MinBufferDuration=8,EnableSync=0 — changed to MinBufferDuration=0.5,AudioLipSyncRatio=0.5,EnableSync=1
per the docs, and dialed in convai.LipSync.AmplitudeMultiplier 0.55 / convai.Emotions.IntensityScale 0.35. This significantly reduced the text-before-speech
delay and toned down viseme exaggeration. Thanks for the docs being clear on this.

Two issues remain on beta.15:

  1. Lip-sync still not fully phoneme-aligned, especially at the start of a response.
  2. The character’s Talk state drifts back to Idle mid-response — she visibly disengages and looks away from camera while still speaking. On a fixed close-up
    shot it’s very noticeable (looks like a puppet snapping to its rest pose mid-performance).

Looking at the SDK release notes, beta.18’s “LipSyncTimeOffset adjusted to 200ms” and beta.19/21’s MetaHuman animation overhaul look like they could be
exactly what we need — but we can’t find instructions for updating the Convai plugin in an existing project (beta.15 → latest). Could you point us to the
update process, and confirm whether beta.21 addresses these two issues for MetaHuman characters specifically?

Screenshots attached: plugin version (beta.15), scene framing, .ini config, console command setup.

Happy to share the Character ID or project files.

What did you expect?

Believable well synced mouth movements. Not overly exaggerated. Also the character must stay engaged with the camera whilst talking and not drop into the Idle animation, making it seem bored and disinterested.

Screenshots, recording, logs, or minimal repro

0-Convai-plugin-version-4-0-0-beta-15.png
0-Convai-plugin-version-4-0-0-beta-15.png2186×310 34.6 KB
4-full-text-block-before-speech.png
4-full-text-block-before-speech.png1024×579 128 KB
5-idle-state-while-talking.png
5-idle-state-while-talking.png1024×578 131 KB
3-dampened-exaggerated-visemes.png
3-dampened-exaggerated-visemes.png2096×1068 171 KB
1-scene-framing-context.png
1-scene-framing-context.png2708×1554 457 KB

I checked the relevant documentation, resources, and similar forum posts before posting.

on

I confirm that I did not include API keys, tokens, passwords, secrets, private character data, private conversation logs, or customer data.

on

2

Please do not use any parameters and follow the documentation.

3

Appreciate you getting back to me Kaan. Thanks, I’ll do as you instruct and will report back with the results.

I only changed the values out of desperation to get the lip-sync to work correctly. Because the out of the box results were terrible and have seriously hampered my ability to show demos to clients.

Fingers crossed this update will fix all.

Thanks,

Pierson