Convai Plugin on Unreal – Audio Cuts Off at End of Conversation (Since June 11)

Unreal Engine Questions
,
62

Dear team,

We have a live event with the government tomorrow, and unfortunately, the characters are still experiencing critical failures. The issue is widespread: the character audio is cutting off, with speech starting normally but then breaking abruptly after the first few words, pausing for several seconds, and then resuming.

This is happening across all characters, with over 10 instances affected, which clearly points to an internal issue within the platform. We’ve also seen other users reporting similar behavior.

We urgently request a response and update on the root cause of this issue and what is being done to resolve it, considering the critical nature of our upcoming official event.

Thank you in advance for your prompt attention.

Sincerely,

63

Unfortunately, the issue occurring with ElevenLabs voices is quite complex, which makes it unlikely that we can release a fix in the short term. Additionally, while you’ve mentioned that this problem has been happening since June, there haven’t been any changes on our side during that time. It’s possible that ElevenLabs made some updates on their end that are affecting API calls and causing this behavior. As a temporary solution, we recommend trying other voice providers to avoid these disruptions.

64

Thank you for your response. However, I’d like to clarify that the character scheduled to be used in an official live event with the government tomorrow is configured with a Google Cloud (GCP) voice. Nevertheless, it is presenting the exact same issue: abrupt audio cuts and unexpected pauses right after starting to speak.

Therefore, it is clear that the issue is not related to ElevenLabs, as suggested, but rather to the platform itself or its integration layer with voice providers.

We emphasize that this is a live institutional event with a fixed schedule, and we urgently need a solution.

65

We haven’t encountered this issue with GCP voices before. It’s possible that the delays are network related, causing the audio to stutter or lag. To help us investigate further, could you please share your logs along with a video recording that demonstrates the problem?

66

Dear team,

As the content of our material is confidential, we kindly ask that you provide a direct and secure email address for submission.

We are experiencing serious issues also with GCP voices, and we have a recorded video clearly showing the problem, which demonstrates that this is not related to ElevenLabs.

Additionally, we have several separate applications outside of Convai using ElevenLabs, and none of them show any instability.

We await the email address so we can send the material, given the urgency of the situation.

67

You can send it as a private message here. Additionally, please note that using ElevenLabs voices via API calls is different from using them directly on the ElevenLabs website.

68

Great, K3! Thanks for the response.

Yes, we’re aware of the differences between using ElevenLabs voices via API and directly on their website. However, I’d like to share that in our case, we’ve been using ElevenLabs via API in other applications and haven’t experienced the kind of issue you’re reporting, such as the audio cutting off at the end of the conversation.

That said, I’m sending you a private message with a video demonstrating the audio cut behavior, captured via GCP, along with the ID of one of the characters showing the issue, in case it helps with your technical investigation.

69

The video you shared is unfortunately unclear, and we can’t determine what’s happening from it. Please provide a new recording where the issue is fully visible and easy to understand. Additionally, kindly share your logs so we can investigate the problem in detail.

By the way, which version of the Convai Plugin are you using?

70

We are having the same issues! Convai Plugin 3.6.3 doesn’t seem to solve it. It’s a very severe problem.
The character is pausing every 1-2 seconds for about 0.5 seconds, while the logs show:
ConvaiAudioStreamerLog: State transition: WaitingOnLipSync → Playing
ConvaiAudioStreamerLog: onAudioFinished
ConvaiAudioStreamerLog: onAudioFinished
ConvaiAudioStreamerLog: onAudioFinished
ConvaiAudioStreamerLog: onAudioFinished
with onAudioFinished being called everytime the hitch happens (once per second)

Changing MinBufferDuration=0 seems to decrease the frequency of when this happens, but when the glitch does happen it’s much more drawn out (happening every frame)
ConvaiAudioStreamerLog: State transition: WaitingOnLipSync → Playing
ConvaiAudioStreamerLog: onAudioFinished
ConvaiAudioStreamerLog: onAudioFinished
ConvaiAudioStreamerLog: onAudioFinished
ConvaiAudioStreamerLog: onAudioFinished
ConvaiAudioStreamerLog: onAudioFinished
ConvaiAudioStreamerLog: onAudioFinished
ConvaiAudioStreamerLog: onAudioFinished
ConvaiAudioStreamerLog: onAudioFinished
ConvaiAudioStreamerLog: onAudioFinished
ConvaiAudioStreamerLog: onAudioFinished
ConvaiAudioStreamerLog: onAudioFinished…etc. many many more instances of this per second

We are using Azure voices

Character ID : bdd4e0c4-86a0-11ef-ae94-42010a7be011 | Session ID : 83dcf90e5637cdce8185830878846e67

Is there any information on when this will be fixed or how to mitigate the issue?? latest plugin updates do not solve.

Also - deleting ConvaiFaceSync seems to stop the issue from occurring

71

Could you please try adding MinBufferDuration=10 under Project Settings > Convai > Advanced > Extra Params?

72

I’m having the same issue with the gpc voice. I’m developing a game in Unity with convai version 3.2.4.

73

Hello @lizzicorleo,

Could you please share your character ID?

74

did you guys change something? Seems to be working fine now even without MinBufferDuration change

75

e890b0fe-2a8d-11f0-a4be-42010a7be01d, Sometimes, the next section does not activate after the player gives the correct answer. You have to repeat it to trigger it. Other times, it responds with half-finished sentences or phrases that are not relevant to the narrative tree.

76

If this is the issue you are experiencing, it is a completely different issue. It may be related to your ND decisions.

77

For example? These are very simple decisions such as “the user must say the number three.” It happened that the user said number three, but the next section did not start. Furthermore, not all sentences are completed, but are interrupted halfway through, as happened in this discussion.

78

This is a different issue. Please create a new post.

1 Like
79

Hi all,
I wanted to provide an update and also raise a critical concern that is currently affecting our production environment.

We’ve tested again with the latest Convai plugin and tried setting MinBufferDuration=10 under Project Settings > Convai > Advanced > Extra Params as suggested. Unfortunately, the issue persists with no noticeable improvement.

Our setup is dependent on a custom ElevenLabs voice, which is essential to our project — not because it mitigates the bug, but because the voice itself is a core part of our application.

In our latest Unreal Engine debug session, the avatar started speaking and then abruptly stopped very early into the dialogue.
Looking at the logs, the final message was:

ConvaiAudioStreamerLog: onAudioFinished

What’s even more concerning is that this session appears empty in the Convai dashboard under Memory. I can see the session ID (24c15723fa8674c724c074e0466dfae0) and the container, but both my prompt and the avatar’s response are completely missing. This might be related to the voice-cutting issue, or it might be a separate backend problem — I’m reporting it here for completeness.

Here’s an excerpt from the full logs:

ConvaiAudioStreamerLog: State transition: WaitingOnAudio -> WaitingOnLipSync
ConvaiAudioStreamerLog: State transition: WaitingOnLipSync -> Playing
...
ConvaiGRPCLog: Received Audio Chunk: 1.724580 secs
ConvaiAudioStreamerLog: State transition: Playing -> WaitingOnLipSync
ConvaiAudioStreamerLog: onAudioFinished

:pushpin: We urgently need help identifying a reliable workaround.
We’ve launched our experience publicly, but for nearly two months the platform has been unstable and unpredictable. I understand the technical complexity, but is there still no clear root cause or fix for this?

It seems I’m not the only one affected (aside from a few unrelated or mixed-in reports).
Are there any extra parameters or settings we could try in Unreal to stabilize this?

Thanks in advance for any support.

Here is the full log

80

Team, we’re in the middle of the event and the character’s audio is failing again. This voice issue is already known, and there are multiple user reports about it.

Yesterday it was working perfectly. Today, without any changes to our setup, it started cutting out. We’re using a dedicated network exclusively for the system, with 300mb upload speed.

The voice is generated via GCP, and as we’ve already clarified, this is not an ElevenLabs issue.

We need an immediate solution or at least a functional contingency right now.

Character ID: 29df0f22-59cf-11f0-b23d-42010a7be01f

Awaiting urgent response.

81

Hello @omar.venturi,

Thank you for sharing the detailed information. I had previously suggested changing the MinBuffer parameter for @Pablo_Lucio, as he was using Azure voices. If you adjust the MinBuffer value, you also need to disable FaceSync but in your case, there’s no need to change anything at this stage.

We’ve been able to reproduce this issue on our end, but unfortunately, it’s a complex, multi-layered problem with no immediate fix available. Our engineering team is investigating and will release an update soon, after which we’ll ask you to test it on your setup.

Additionally, I’ve removed the log you shared since it contained your API Key. As mentioned earlier, please make sure to redact any sensitive information before posting logs in the forum.

We’ll keep you updated as soon as we have an update.