2786b252-9d59-11f0-88d8-42010a7be025 and all other characters who are using Convai SDK 4.4.0
Where does it happen?
Other
Language
All languages
Voice provider
Eleven Labs, OpenAI
Voice name
Ophelia EL is the voice of the character ID given but happens with any voice
Environment
Meta Quest 3, Convai SDK 4.4.0
Short summary
Characters keep getting cut off mid sentence
User input
Hi, tell me about your day
What happened?
characters on the realtime pipeline (Convai SDK v.4) keep cutting out mid sentence and I’ve logged a session that shows why. Four times in three minutes (09:55:13, 09:55:54, 09:56:24 and 09:57:02 in the attached log taken 5 mins ago) the voice detection reported that I’d started speaking for just 10 to 20 milliseconds with the transcription coming back empty, but I wasn’t speaking, and each time it interrupted the character mid reply, then generated a new reply to that phantom “speech”, which sometimes got cut off the same way 0.2 seconds after it started. When I genuinely talk over the character the log shows seconds of detected speech with a proper transcription so real interruptions look completely different from these false ones. Also her replies also have short audio dropouts of up to a second mid sentence even when nothing interrupts them (the relevant lines are tagged V4ChopDiag). I have several Convai apps and the one’s that still uses the older V3 pipeline is completely unaffected so my hunch is something has changed in the realtime pipeline. This has gone on for a while now (week or so) and I was kinda hoping it would solve itself as I don’t like to bother you guys but users are complaining now (and rightly so, it’s a fine balance between acting on everything that is said i.e. too much and not acting soon enough, which I have done here unfortunately). Most of the characters use Gemma 4 31B or 26B LLM’s, with mostly EL voices and a few with OpenAI voices. I changed the LLM to Grok 4.3 for the character ID given here and it seemed a bit better but it could have been a co-incidence.
What did you expect?
For the character to tell the user about their day.
Can you use the latest version if possible?
For a quick fix, you can follow the steps.
Go to Convai Manager → Advanced Room Control → Barge In → Client Speech Detection and select Disable in place of Silero.
I’ve done more digging with your suggested fix applied and found something concrete: Client Speech Detection was already Disabled in my integration, and I also tried sending explicit vad_params (confidence 0.8, start_secs 0.3) but there was no change. The reason is in the logs: your pipeline sends my app a false “user started speaking” event every 30.1 seconds, almost as though on a timer, with timestamps 42.4, 72.6, 102.8, 132.9, 163.0, 193.1, 223.2, 253.3 in my latest session and the same 30 second pattern is in all three logs I’ve captured since the yesterday. Each event lasts 0 to 20ms with an empty transcription and whenever one lands while the character is speaking it interrupts her mid sentence (interrupted=true) then the reply that regenerates gets cut by the next tick. Real interruptions (me actually talking over her) show up completely differently by seconds of speech, full transcription. Separately, her replies also contain audio dropouts of 30 to 950ms with no interruption events at all so the stream delivery itself is also ragged. I’ve uploaded my log files so you can verify.
Quick update : I have reverted to SDK version 3 for the time being as this has smooth conversation, so please don’t consider my request as urgent etc, the status is no longer “Blocker - I cannot continue”.
Hi Kaan, i will test a sample character pretty soon no problem, right now I am using the old gRPC pipeline again so it will require a code change which I will do. I am referring to the REST API endpoint method of client/server communications in the 4.4.0 SDK, i was doing a gradual update of the SDK because of the complexity of doing it all at once, i appreciate this is probably deemed not advisable but i think just starting again from scratch would overwhelm me lol. These interruptions were happening with all 18 of my characters. I do wonder if its a fault on my side because i see no other complaints about it here on the forum.
