Lipsync - Text to Speech

Unreal Engine Questions
1

Original Discord Post by bobbybay | 2023-12-05 09:20:10

Hello, I’m trying to build a way of getting the convai character to output a pre-recorded .wav file with lipsync. It currently works by generating text to speech and playing it back. But I can’t get the OVRLipSync to work with pre-recorded audio files. Any ideas? Thanks a bunch!

Images:

image.png
image.png1464×471 78.7 KB

2

Reply by k3kalinix | 2023-12-05 09:30:34

Hello <@728120809712779315>, Welcome to Convai Community!

3

Reply by k3kalinix | 2023-12-05 09:30:43

<@808164597050638367> Could you help?

4

Reply by bobbybay | 2023-12-05 09:35:10

Hi <@1023671043287699568> Thanks for the quick reply and help. Much appreciated.

5

Reply by k3kalinix | 2023-12-05 09:37:16

You are welcome!

6

Reply by freezfast | 2023-12-05 09:49:59

Hi <@728120809712779315>, we currently do not support lipsync with pre-recorded audio / text to speech function at the moment. However, after updating to our latest plugin you should be able to use Invoke Speech instead to get the character to say what you need him to say.
cc: <@808164597050638367> for a quick guide

7

Reply by bobbybay | 2023-12-05 09:57:15

Thanks <@365628745886859267> . I assume Invoke Speech is still using the API / an internet connection to do the text to speech? I was looking for a solution when the application loses a connection to the internet for whatever reason, as a fallback option. I’ll look into the invoke speech whatsover. Is the plugin updated on the GDrive link too? Or only though the Epic Marketplace? Thanks

8

Reply by freezfast | 2023-12-05 09:58:29

I see, it is updated on both and we no longer use ConvaiOVRLipSync, <@808164597050638367> will refer you to the corresponding guides

9

Replying to bobbybay’s Message

Reply by bobbybay | 2023-12-05 09:57:15
Thanks <@365628745886859267> . I assume Invoke Speech is still using the API / an internet connection to do the text to speech? I was looking for a solution when the application loses a connection to the internet for whatever reason, as a fallback option. I’ll look into the invoke speech whatsover. Is the plugin updated on the GDrive link too? Or only though the Epic Marketplace? Thanks

Reply by mrd7041 | 2023-12-05 10:01:06

Hello <@728120809712779315> you can please refer to this guide for adding the new lipsync component called Convai Face Sync.
Link: - Adding LipSync to MetaHuman (From plugin version 3.0.0 or later ) | Documentation

Embedded Content:
Adding LipSync to MetaHuman (From plugin version 3.0.0 or later )
Add lip sync to your MetaHuman from base plugin
Link: Adding LipSync to MetaHuman (From plugin version 3.0.0 or later ) | Documentation
Thumbnail

10

Reply by bobbybay | 2023-12-05 10:03:35

Thanks a bunch guys, I completely missed this update. I will try and update my project to the new Convai version. Cheers

11

Replying to bobbybay’s Message

Reply by bobbybay | 2023-12-05 10:03:35
Thanks a bunch guys, I completely missed this update. I will try and update my project to the new Convai version. Cheers

Reply by k3kalinix | 2023-12-05 10:05:42

If you have any other questions, please do not hesitate to ask.
We would be very grateful if you could leave a comment on our store pages with your company name or the name of your project. We value your feedback and comments as they help us improve our services and products.
Asset Store: NPC AI Engine - Dialog, actions, voice and lipsync - Convai | Behavior AI | Unity Asset Store
Unreal Engine: https://www.unrealengine.com/marketplace/en-US/product/convai/reviews

12

This conversation happened on the Convai Discord Server, so this post will be closed.