[UE 5.5 | V4 Beta] Issues with Push-to-Talk

Unreal Engine Questions
1

Hello Convai Team,

I’ve recently migrated a project from the stable Marketplace plugin (v3) to the V4 Beta (4.0.0-beta.4) to utilize the new Vision capabilities in Unreal Engine 5.5. After following the setup guides and swapping to the new native components, I’m encountering a few critical issues that I haven’t been able to solve after searching the forums and documentation.

My setup is a “disembodied” architecture:

  • BP_FirstPersonCharacter: Contains the Convai Player and Environment Webcam components.

  • BP_Vivian1 (NPC): Contains the Convai Chatbot component.

Problem 1: Push-to-Talk is Not Working (Stuck in Hands-Free Mode)

Expected Behavior:
The microphone should only activate for transcription when I press and hold a key (e.g., ‘T’).

Actual Behavior:
The microphone is “always-on.” The logs show user-started-speaking events even when I am not pressing the key. This leads to incorrect transcriptions (picking up its own audio or background noise) and makes Push-to-Talk impossible.

What I Have Tried So Far:

  1. Details Panel: I’ve searched the Details panel of the Convai Player component for “Enable Voice Activity Detection,” “VAD,” and “Voice,” but the setting does not appear in this beta version, so I cannot disable it via the editor UI.

  2. Blueprint Override: I tried forcing VAD off via Blueprint on BeginPlay using Set Enable Voice Activity Detection, but this function does not seem to exist for the Convai Player component.

  3. Mute/Unmute: I attempted to implement PTT by calling Mute Streaming Audio on BeginPlay and Unmute Streaming Audio on key press. This resulted in the log repeatedly printing Warning: UnmuteStreamingAudio: already streaming!, indicating that the Mute function was not effective.

  4. Start/Stop Listening: I searched for Start Listening and Stop Listening functions on the Convai Player component, but they do not appear to exist in this version.

    Here is the log from the unreal engine ,


LogBlueprintUserMessages: [MicSettings_WB_C_0] Device Set Succesfully
ConvaiSubsystemLog: OnConnectedToServer called
ConvaiReferenceAudioThread: ConvaiReferenceAudioThread created with chunk size: 480
ConvaiReferenceAudioThread: ConvaiReferenceAudioThread initialized
ConvaiReferenceAudioThread: ConvaiReferenceAudioThread started running
ConvaiReferenceAudioThread: Reference audio capture started
ConvaiSubsystemLog: Started reference audio capture thread
ConvaiPlayerLog: Server connection state changed: 2
ConvaiPlayerLog: Server connected and auto-initialize enabled - starting session
ConvaiPlayerLog: Started Streaming Audio
ConvaiAudioLog: Using Microphone Array (Realtek(R) Audio) as Audio capture device with NumChannels:2 and SampleRate:48000
LogAudioCaptureCore: Display: WasapiCapture AudioFormat SampeRate: 48000, BitDepth: 32-Bit Floating Point
ConvaiPlayerLog: Started default audio capture
ConvaiSubsystemLog: 🔌 Attendee connected: ConvAI-Bot
ConvaiSubsystemLog: Attendee ID: ConvAI-Bot, Data: {"data":{"config":[],"version":"1.0.0"},"id":"","label":"rtvi-ai","type":"bot-ready"}
1 Like
2

Things are working fine on our end. Could you please try creating a new project and test again? and kindly share more detailed logs. If possible, you can also share your project with us so we can investigate further.