The character occasionally fails to respond

Core Features Conversation Issues
1

Hi :slight_smile:

For a good while the characters in my Unity app for Quest 3 VR headsets have been failing to respond to the user now and again, at a very rough guess I’d say the character fails to respond once in about 12-15 responses but it varies. I’ve ignored it up to now but decided to look at it more closely today to try to eradicate it because it is off-putting.

I enabled all the Convai Logger Settings in Unity and noticed this output to my log file when the app runs, when the failed response happens :

10:42:33.992 <color=grey>[Info][Character]: Hannah is now listening</color>
[Stack Trace - Method: RunInternal, at Line: 0 in File: ]
10:42:36.970 <color=grey>[Info][Character]: Microphone is working fine.</color>
[Stack Trace - Method: InvokeMoveNext, at Line: 0 in File: ]
10:42:46.047 RpcException: Status(StatusCode=Internal, Detail="Get Response Failed None Stream removed")
10:42:46.047 Convai.Scripts.Runtime.Core.ConvaiGRPCAPI.ProcessAudioChunk (Grpc.Core.AsyncDuplexStreamingCall`2[TRequest,TResponse] call, System.Int32 diff, System.Collections.Generic.IReadOnlyList`1[T] audioData) (at <00000000000000000000000000000000>:0)
System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine] (TStateMachine& stateMachine) (at <00000000000000000000000000000000>:0)
Convai.Scripts.Runtime.Core.ConvaiGRPCAPI.ProcessAudioChunk (Grpc.Core.AsyncDuplexStreamingCall`2[TRequest,TResponse] call, System.Int32 diff, System.Collections.Generic.IReadOnlyList`1[T] audioData) (at <00000000000000000000000000000000>:0)
Convai.Scripts.Runtime.Core.ConvaiGRPCAPI.ProcessAudioContinuously (Grpc.Core.AsyncDuplexStreamingCall`2[TRequest,TResponse] call, System.Int32 recordingFrequency, System.Int32 recordingLength, UnityEngine.AudioClip audioClip) (at <00000000000000000000000000000000>:0)
System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) (at <00000000000000000000000000000000>:0)
System.Runtime.CompilerServices.AsyncMethodBuilderCore+MoveNextRunner.Run () (at <00000000000000000000000000000000>:0)
UnityEngine.UnitySynchronizationContext+WorkRequest.Invoke () (at <00000000000000000000000000000000>:0)
UnityEngine.UnitySynchronizationContext.Exec () (at <00000000000000000000000000000000>:0)
--- End of stack trace from previous location where exception was thrown ---
Convai.Scripts.Runtime.Core.ConvaiGRPCAPI.ProcessAudioContinuously (Grpc.Core.AsyncDuplexStreamingCall`2[TRequest,TResponse] call, System.Int32 recordingFrequency, System.Int32 recordingLength, UnityEngine.AudioClip audioClip) (at <00000000000000000000000000000000>:0)
System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) (at <00000000000000000000000000000000>:0)
System.Runtime.CompilerServices.AsyncMethodBuilderCore+MoveNextRunner.Run () (at <00000000000000000000000000000000>:0)
UnityEngine.UnitySynchronizationContext+WorkRequest.Invoke () (at <00000000000000000000000000000000>:0)
UnityEngine.UnitySynchronizationContext.Exec () (at <00000000000000000000000000000000>:0)
--- End of stack trace from previous location where exception was thrown ---
Convai.Scripts.Runtime.Core.ConvaiGRPCAPI.StartRecordAudio (Service.ConvaiService+ConvaiServiceClient client, System.Boolean isActionActive, System.Boolean isLipSyncActive, System.Int32 recordingFrequency, System.Int32 recordingLength, System.String characterID, Service.ActionConfig actionConfig, Service.FaceModel faceModel, System.String speakerID) (at <00000000000000000000000000000000>:0)
System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) (at <00000000000000000000000000000000>:0)
System.Runtime.CompilerServices.AsyncMethodBuilderCore+MoveNextRunner.Run () (at <00000000000000000000000000000000>:0)
System.Threading.Tasks.AwaitTaskContinuation.RunCallback (System.Threading.ContextCallback callback, System.Object state, System.Threading.Tasks.Task& currentTask) (at <00000000000000000000000000000000>:0)
System.Threading.Tasks.Task.FinishContinuations () (at <00000000000000000000000000000000>:0)
System.Threading.Tasks.Task.Finish (System.Boolean bUserDelegateExecuted) (at <00000000000000000000000000000000>:0)
System.Threading.Tasks.Task.TrySetException (System.Object exceptionObject) (at <00000000000000000000000000000000>:0)
System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[TResult].SetException (System.Exception exception) (at <00000000000000000000000000000000>:0)
Convai.Scripts.Runtime.Core.ConvaiGRPCAPI.ProcessAudioContinuously (Grpc.Core.AsyncDuplexStreamingCall`2[TRequest,TResponse] call, System.Int32 recordingFrequency, System.Int32 recordingLength, UnityEngine.AudioClip audioClip) (at <00000000000000000000000000000000>:0)
System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) (at <00000000000000000000000000000000>:0)
System.Runtime.CompilerServices.AsyncMethodBuilderCore+MoveNextRunner.Run () (at <00000000000000000000000000000000>:0)
UnityEngine.UnitySynchronizationContext+WorkRequest.Invoke () (at <00000000000000000000000000000000>:0)
UnityEngine.UnitySynchronizationContext.Exec () (at <00000000000000000000000000000000>:0)
--- End of stack trace from previous location where exception was thrown ---
Convai.Scripts.Runtime.Core.ConvaiNPC.StartListening () (at <00000000000000000000000000000000>:0)
System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) (at <00000000000000000000000000000000>:0)
System.Runtime.CompilerServices.AsyncMethodBuilderCore+MoveNextRunner.Run () (at <00000000000000000000000000000000>:0)
System.Threading.Tasks.AwaitTaskContinuation.RunCallback (System.Threading.ContextCallback callback, System.Object state, System.Threading.Tasks.Task& currentTask) (at <00000000000000000000000000000000>:0)
System.Threading.Tasks.Task.FinishContinuations () (at <00000000000000000000000000000000>:0)
System.Threading.Tasks.Task.Finish (System.Boolean bUserDelegateExecuted) (at <00000000000000000000000000000000>:0)
System.Threading.Tasks.Task.TrySetException (System.Object exceptionObject) (at <00000000000000000000000000000000>:0)
System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[TResult].SetException (System.Exception exception) (at <00000000000000000000000000000000>:0)
Convai.Scripts.Runtime.Core.ConvaiGRPCAPI.StartRecordAudio (Service.ConvaiService+ConvaiServiceClient client, System.Boolean isActionActive, System.Boolean isLipSyncActive, System.Int32 recordingFrequency, System.Int32 recordingLength, System.String characterID, Service.ActionConfig actionConfig, Service.FaceModel faceModel, System.String speakerID) (at <00000000000000000000000000000000>:0)
System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) (at <00000000000000000000000000000000>:0)
System.Runtime.CompilerServices.AsyncMethodBuilderCore+MoveNextRunner.Run () (at <00000000000000000000000000000000>:0)
System.Threading.Tasks.AwaitTaskContinuation.RunCallback (System.Threading.ContextCallback callback, System.Object state, System.Threading.Tasks.Task& currentTask) (at <00000000000000000000000000000000>:0)
System.Threading.Tasks.Task.FinishContinuations () (at <00000000000000000000000000000000>:0)
System.Threading.Tasks.Task.Finish (System.Boolean bUserDelegateExecuted) (at <00000000000000000000000000000000>:0)
System.Threading.Tasks.Task.TrySetException (System.Object exceptionObject) (at <00000000000000000000000000000000>:0)
System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[TResult].SetException (System.Exception exception) (at <00000000000000000000000000000000>:0)
Convai.Scripts.Runtime.Core.ConvaiGRPCAPI.ProcessAudioContinuously (Grpc.Core.AsyncDuplexStreamingCall`2[TRequest,TResponse] call, System.Int32 recordingFrequency, System.Int32 recordingLength, UnityEngine.AudioClip audioClip) (at <00000000000000000000000000000000>:0)
System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) (at <00000000000000000000000000000000>:0)
System.Runtime.CompilerServices.AsyncMethodBuilderCore+MoveNextRunner.Run () (at <00000000000000000000000000000000>:0)
UnityEngine.UnitySynchronizationContext+WorkRequest.Invoke () (at <00000000000000000000000000000000>:0)
UnityEngine.UnitySynchronizationContext.Exec () (at <00000000000000000000000000000000>:0)
--- End of stack trace from previous location where exception was thrown ---
System.Runtime.CompilerServices.AsyncMethodBuilderCore+<>c.<ThrowAsync>b__7_0 (System.Object state) (at <00000000000000000000000000000000>:0)
UnityEngine.UnitySynchronizationContext+WorkRequest.Invoke () (at <00000000000000000000000000000000>:0)
UnityEngine.UnitySynchronizationContext.Exec () (at <00000000000000000000000000000000>:0)

10:42:48.385 <color=grey>[Info][Character]: Hannah has stopped listening</color>
[Stack Trace - Method: StopListening, at Line: 0 in File: ]

(Sorry for the wall of text)

This character is 12c5d804-bddd-11ef-905b-42010a7be016 and the failed responses happen with all characters.

2

Hello @Tyke,

Thanks for the details!

Does it also happen in the playground? If so, could you please share the session ID?

3

Hi K3, I’ve just chatted in the Playground just now and it didn’t happen. Not sure what to do about it to be honest.

4

So it only happens on the Unity side?

5

I asked the avatar about 15 questions in the playground and they replied every time. So yes it appears to only happen in Unity. Forum member Bob Hawkey’s Unity app says his has the same thing happening with occasional failed responses in fact I’ve just seen it happen today or yesterday in 1 of his video’s. I’m not sure if the same console error is generated but the way it failed is the same as in my app.

6

Hello @Tyke,

This is a known issue. We recommend waiting for our upcoming new SDK release.

1 Like
7

Sorry to hijack the topic, we’re having the same issue as reported by Tyke. It occurs randomly, making our user data collection process difficult.

Are there any known workarounds other than handling the exception client-side?

8

Could you please share the logs?

9

This is from our development build log. Today is the first time this has happened (and it happened several times consequently). This happened during long conversation sessions (~20 minutes).

Keep in mind we had to customize the Convai Unity client to support our specific use case (multi-scene setup, PCVR target, and client-side conversation logging).

This is pretty off-putting for the Player. They see the transcript in real time in the chatui but never receive an answer from the character.

(Filename: C:/Users/USER/Documents/Unity/counseling/Assets/Convai/Scripts/Runtime/LoggerSystem/ConvaiLogger.cs Line: 135)

<color=yellow>[Warning][Character]: RPC error during audio chunk processing: Internal - Status(StatusCode=Internal, Detail="Get Response Failed None Stream removed")</color>
[Stack Trace - Method: MoveNext, at Line: 926 in File: C:\Users\USER\Documents\Unity\counseling\Assets\Convai\Scripts\Runtime\Core\ConvaiGRPCAPI.cs]
UnityEngine.StackTraceUtility:ExtractStackTrace () (at C:/build/output/unity/unity/Runtime/Export/Scripting/StackTrace.cs:37)
UnityEngine.DebugLogHandler:LogFormat (UnityEngine.LogType,UnityEngine.Object,string,object[])
UnityEngine.Logger:Log (UnityEngine.LogType,object)
UnityEngine.Debug:LogWarning (object)
Convai.Scripts.Runtime.LoggerSystem.ConvaiLogger:LogMessage (string,Convai.Scripts.Runtime.LoggerSystem.ConvaiLogger/LogLevel,Convai.Scripts.Runtime.LoggerSystem.ConvaiLogger/LogCategory,object[]) (at C:/Users/USER/Documents/Unity/counseling/Assets/Convai/Scripts/Runtime/LoggerSystem/ConvaiLogger.cs:90)
Convai.Scripts.Runtime.LoggerSystem.ConvaiLogger:Warn (string,Convai.Scripts.Runtime.LoggerSystem.ConvaiLogger/LogCategory,object[]) (at C:/Users/USER/Documents/Unity/counseling/Assets/Convai/Scripts/Runtime/LoggerSystem/ConvaiLogger.cs:135)
Convai.Scripts.Runtime.Core.ConvaiGRPCAPI/<ProcessAudioChunk>d__47:MoveNext () (at C:/Users/USER/Documents/Unity/counseling/Assets/Convai/Scripts/Runtime/Core/ConvaiGRPCAPI.cs:926)
System.Runtime.CompilerServices.AsyncTaskMethodBuilder:Start<Convai.Scripts.Runtime.Core.ConvaiGRPCAPI/<ProcessAudioChunk>d__47> (Convai.Scripts.Runtime.Core.ConvaiGRPCAPI/<ProcessAudioChunk>d__47&)
Convai.Scripts.Runtime.Core.ConvaiGRPCAPI:ProcessAudioChunk (Grpc.Core.AsyncDuplexStreamingCall`2<Service.GetResponseRequest, Service.GetResponseResponse>,int,single[])
Convai.Scripts.Runtime.Core.ConvaiGRPCAPI/<ProcessAudioContinuously>d__45:MoveNext () (at C:/Users/USER/Documents/Unity/counseling/Assets/Convai/Scripts/Runtime/Core/ConvaiGRPCAPI.cs:771)
System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner:InvokeMoveNext (object)
System.Threading.ExecutionContext:RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool)
System.Threading.ExecutionContext:Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool)
System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner:Run ()
System.Threading.Tasks.SynchronizationContextAwaitTaskContinuation/<>c:<.cctor>b__7_0 (object)
UnityEngine.UnitySynchronizationContext/WorkRequest:Invoke () (at C:/build/output/unity/unity/Runtime/Export/Scripting/UnitySynchronizationContext.cs:153)
UnityEngine.UnitySynchronizationContext:Exec () (at C:/build/output/unity/unity/Runtime/Export/Scripting/UnitySynchronizationContext.cs:83)
UnityEngine.UnitySynchronizationContext:ExecuteTasks () (at C:/build/output/unity/unity/Runtime/Export/Scripting/UnitySynchronizationContext.cs:107)
10

Could you please share the Session ID?