ElevenLabs Voice audio response via convai much more buggy than via ElevenLabs

Core Features Language and Speech
1

Hello convai Team,

we are using convai via Unity for a character which uses a eleven labs voice (custom female).
The generated audio when testing ( -in App- but also via -web- > convai/playground/character/language-and-voice) has much more artifacts and bugs regarding speed and tone of voice than audio created directly via eleven labs website.
The bugs were mentioned before in other forum posts ( ElevenLabs voice returning “slow motion” audio - Core Features / Language and Speech - Convai Developer Forum) … the audio is streched, the gaps between words and general speed of speaking is sometimes super fast and in the same audio but another part of a sentence it becomes super slow etc. … these bugs appear super frequent when audio is generated through convai but only very rarely when created directly via eleven labs and its super frustrating since we dont see an obvious reason. In another forum post, I already addressed that voice parameters and voice AI model are being checkt and this seems to be correct now (as far as I can imagine … since there is no user based configuration UI).

In addition to the speed of generated voice audio we sometimes get cut-off audios where only 20%-40% of the original text output gets generated as audio. Which is of course a black-box bug to us, too.

So overall - the audio voice of convai character is simply way too bad to be used in a professional project - and the only one to change this is your team.
How can I help to fix and challenge these issues or do you do this on purpose for what ever business reason?

cheers,
Markus

2

Hello @info_AM,

Could you please share a comparison video (Elevenlabs/Unity-Convai Side)? Also please make sure you are using our latest Unity SDK, there are some fixes for Elevenlabs voices.

3

Took me some time - due to project load - but here we go.

1 Like
4

Thanks @info_AM !

5

both - the slowmotion audio bug and the bug where audio answers get cut off are super annoying and only happen if I use elevenlabs via convai … if I use it directly via elevenlabs website I dont get these issues.

Both issues also appear, when we use convai in our unity project …

6

My characters that use Eleven Labs through Convai sometimes get cut off as well.

7

Hello @Tyke,

Make sure you are using the latest version of the Unity SDK.

8

@info_AM Which version of the Unity Convai SDK are you using?

9

I’m not unfortunately, I have edited quite a few Convai scripts so I’m a bit reluctant to update, I don’t suppose there is any relevant code I can pinpoint so I can just copy and paste the new code into my existing scripts? I don’t imagine that is easy etc so no problems if not. I know I’ll have to update at some point.

11

First create a new project and test it there.

12

I think we are using 3.3.0.