Lip Sync in Italian

[novahuman]

Hello,

We’re using convai plugin in unreal, and we noticed that, especially in italian, the lipsync is not accurate at all.. it just generates random mouth moviments.

There is any way to improve it?

[K3]

Hello @novahuman,

This could be related to the specific voice you’re using. Could you please share which voice you’re currently using and the Character ID?

[novahuman]

ID: 7d6a442e-069e-11f0-b700-42010a7be01a

RyanMultilingual Male American English Voice

[novahuman]

I tested different voices - included Eleven Labs and still facing the same issue

[K3]

@novahuman, Could you please try testing your character using the Video Call feature on convai.com? Just click on your character, then use the video call icon on the right side (character thumbnail).

[K3]

So, I would say it worked as expected.

[novahuman]

Did you make some fix on the male character? and - this will affect also the plugin in unreal?

[novahuman]

This is the same character ID in unreal - and as you can see here the lipsync is not accurate at all.

[K3]

Thanks for your update.

Just to clarify, the Avatar Studio on convai.com uses client-side lip sync, which provides the best visual accuracy. However, the Unreal Engine plugin and SDKs rely on server-based lip sync, which may not achieve the same quality, especially with non-English utterances.

[novahuman]

There is any way to improve the lip-sync in unreal engine that you can advice?

[K3]

Unfortunately, no.