AI assistant interrupted during playback

What do you need help with?

Bug report

Installation method

Manual installation / GitHub

Issue area

Connection / WebRTC

Impact

Major - a core flow is broken

Unreal Engine version

5.8.0

Convai Unreal Engine Plugin version

4.0.0-beta.24

Project setup

Blueprint

Target platform

Unreal Editor

Where does it happen?

Custom project

Short summary

During playback of an already received response, the AI assistant is unexpectedly interrupted approximately halfway through the sentence.

What happened?

During playback of an already received response, the AI assistant is unexpectedly interrupted approximately halfway through the sentence, even though there is no new response or user input at that moment. At the same time, the On Started Talking and On Finished Talking events are triggered in the log. After that, the current speech stops, but the character does not begin saying anything new. Could you please check why these events are triggered during active speech and whether they may be related to the interruption?

Unreal logs or error messages

LogBlueprintUserMessages: [WBP_Main_UI_C_0] START TALKING
ConvaiSubsystemLog: :microphone: Active speaker changed: convai-bot
ConvaiSubsystemLog: Attendee ID: convai-bot, Data: {“epoch”:1,“label”:“rtvi-ai”,“neurosync_turn_id”:4,“response_id”:“1a88b6ac-560c-4d49-a98e-4436a8575a9d:r6”,“sequence”:60,“type”:“bot-stopped-speaking”}
ConvaiChatbotComponentLog: UConvaiChatbotComponent Transcription received: | Character ID : 0dfaadf8-5432-11f1-8aa0-42010a7be02e | Session ID : -1
ConvaiSubsystemLog: Attendee ID: convai-bot, Data: {“epoch”:1,“neurosync_turn_id”:4,“response_id”:“1a88b6ac-560c-4d49-a98e-4436a8575a9d:r6”,“sequence”:61,“type”:“bot-turn-completed”,“was_interrupted”:true}
ConvaiSubsystemLog: Warning: OnDataPacketReceived: Unknown packet type ‘bot-turn-completed’.
ConvaiSubsystemLog: Attendee ID: convai-bot, Data: {“data”:{“bt_code”:“”,“bt_constants”:“”,“narrative_section_id”:“d263c07c-55d0-11f1-b153-42010a7be02e”,“type”:“behavior-tree-response”},“label”:“rtvi-ai”,“type”:“server-message”}
ConvaiSubsystemLog: Attendee ID: convai-bot, Data: {“label”:“rtvi-ai”,“type”:“user-started-speaking”}
ConvaiChatbotComponentLog: InterruptSpeech: Interrupting character | Character ID : 0dfaadf8-5432-11f1-8aa0-42010a7be02e | Session ID : -1
ConvaiChatbotComponentLog: onAudioFinished - IsConnectionTalking: false
ConvaiFaceSyncLog: ConvaiPauseLipSync: Pausing lipsync - TotalFrames: 20, FramesRemaining: 9, TotalDuration: 0.333, TimePlayed: 0.179, TimeRemaining: 0.154, TotalPausedDuration: 0.000
ConvaiPlayerLog: Started talking
LogBlueprintUserMessages: [WBP_Main_UI_C_0] FINISH TALKING
LogBlueprintUserMessages: [WBP_Main_UI_C_0] “ArmHomeSpeakReminder: HOME, Armed=true, starting timer”
LogBlueprintUserMessages: [WBP_Main_UI_C_0] ResetInactivityTimer called
LogBlueprintUserMessages: [WBP_Main_UI_C_0] ResetInactivityTimer ->120.0
ConvaiFaceSyncLog: ConvaiStopLipSync: Stopping lipsync - bIsPlaying: true, bIsPaused: true, TotalFrames: 20, FramesRemaining: 9, TotalDuration: 0.333, TimePlayed: 0.179, TimeRemaining: 0.154, TotalPausedDuration: 0.000
ConvaiChatbotComponentLog: Character interrupted by user speaking
ConvaiSubsystemLog: Attendee ID: convai-bot, Data: {“label”:“rtvi-ai”,“type”:“user-stopped-speaking”}
ConvaiChatbotComponentLog: Character interrupt ended
ConvaiSubsystemLog: Attendee ID: convai-bot, Data: {“data”:{“event_type”:“trigger-message”,“extras”:{“actual_run_llm”:“true”,“bot_state”:“speaking”,“downgrade_reason”:null,“downgraded”:false,“has_speak_tag”:false,“llm_triggered”:t

Screenshots, recording, logs, or minimal repro

Log_UE_ConvAI_Receive_StartFinish_Events.txt (1.1 MB)

I checked the relevant documentation, resources, and similar forum posts before posting.

on

I confirm that I did not include API keys, tokens, passwords, secrets, private character data, private conversation logs, or customer data.

on

Are you using Hands Free mode?

Also, does the same issue occur when you test with Push to Talk mode?

No, I am not using Hands Free mode. I am currently using Push to Talk.

Could you please test it on Avatar Studio? Are you seeing the same issue there?