Lip Sync not working in two languages case

Unity Questions
1

Original Discord Post by osamakhattak10 | 2024-08-03 11:27:02

I am trying to use Convai in unity and I am using two different languages to interact with that include English and Arabic. Now if there is only one language then the Lip Sync works perfectly fine but if I try to use both, the lip sync is not working for any of the language.

2

Reply by osamakhattak10 | 2024-08-03 11:52:41

<@1023671043287699568>

3

Reply by k3kalinix | 2024-08-03 13:17:30

Hello <@1161957269794213898>,

Welcome to Convai Community!

Could you please try with different voice?

Also, could you share your Character ID and session ID?

4

Reply by osamakhattak10 | 2024-08-03 13:30:09

Hii, I have tried with all the voices available for Arabic and English voices.

5

Reply by osamakhattak10 | 2024-08-03 13:30:27

80d62f20-3e98-11ef-9ef6-42010a7be011

Here is my character if

6

Reply by osamakhattak10 | 2024-08-03 13:30:28

id

7

Reply by osamakhattak10 | 2024-08-03 13:31:05

a05136621a877158ccc63f4b5ea04c5d

And this is my latest session ID

8

Reply by k3kalinix | 2024-08-03 13:35:44

Thanks for the details.
I informed my teammates.
We will get back to you (on working day)

9

Reply by osamakhattak10 | 2024-08-05 07:18:52

Hi Good Morning <@1023671043287699568> , Hope you are doing great. I am hoping to get a solution to my problem from you today. Thank you.

10

Reply by k3kalinix | 2024-08-05 16:48:59

Hey <@1161957269794213898>,

As I said before, we will keep you informed.

I cannot share an estimated time.

11

Reply by enormousvampire | 2024-08-06 06:33:17

Heyy <@1161957269794213898> , I did try multilingual voice (Ryan multilingual for my case) and used English, Hindi and German, and LipSync did work for all the languages.

12

Reply by enormousvampire | 2024-08-06 06:33:21

I would suggest you to try with other voices too

13

Reply by enormousvampire | 2024-08-06 06:33:58

Meanwhile I am looking at the issue with your voice. (We did have a bug with OpenAI Voice breaking LipSync, which has been solved internally)

14

Reply by enormousvampire | 2024-08-06 06:34:43

Here is a reference video for English/Hindi audio and lipsync

Attachments:

15

Replying to enormousvampire’s Message

Reply by enormousvampire | 2024-08-06 06:34:43
Here is a reference video for English/Hindi audio and lipsync

Reply by osamakhattak10 | 2024-08-07 08:11:48

Hi <@614184022121119745> , I actually had a detailed look at the issue, It’s happening only with the OpenAI voices. I tested only one language with the OpenAI voices, and the lip sync was not working.

16

Replying to enormousvampire’s Message

Reply by enormousvampire | 2024-08-06 06:34:43
Here is a reference video for English/Hindi audio and lipsync

Reply by osamakhattak10 | 2024-08-07 08:12:17

That’s actually great, can we see an update real soon?

17

Reply by enormousvampire | 2024-08-07 14:14:32

So yeah like I mentioned, we already solved this issue internally. A few code and proto changes

18

Reply by enormousvampire | 2024-08-07 14:14:53

If you wish, I can share with you those particular changes, which will fix this issue

19

Replying to enormousvampire’s Message

Reply by enormousvampire | 2024-08-07 14:14:53
If you wish, I can share with you those particular changes, which will fix this issue

Reply by osamakhattak10 | 2024-08-08 06:03:11

Yes please, that will help

20

Reply by osamakhattak10 | 2024-08-09 12:14:40

Hi <@614184022121119745> , Can I have the update please?