Environment
Unity
6000.4
Convai SDK
4.0.0 (com.convai.convai-sdk-for-unity)
Render Pipeline
URP 17.4.0
Platform
Windows x86-64
LiveKit FFI DLL
Manually replaced with livekit_ffi 0.12.47 (auto-downloader fetched incompatible 0.12.43)
Problem Summary
A scene has 5 ConvaiCharacter components on separate GameObjects under one parent (Carousel_Stage). The player selects one character at a time from a UI carousel. The expected behaviour is that only the selected character responds to voice input. There are three compounding bugs blocking this.
Bug 1 β EnableRemoteTrackRequest Missing from Native FFI DLL
Symptom
Every time a ConvaiCharacter with _enableRemoteAudio = true joins the room, this exception fires:
Exception: Unknown request type: LiveKit.Proto.EnableRemoteTrackRequest
LiveKit.Internal.FFIClients.Requests.FfiRequestWrap`1[T].Send()
at FfiRequestWrap.cs:57
LiveKit.IRemoteTrack.SetEnabled(bool enabled)
at Track.cs:59
Convai.Infrastructure.Networking.Native.NativeRemoteAudioTrack.SetRemoteAudioEnabled(bool)
at NativeRemoteAudioTrack.cs:28
Convai.Infrastructure.Networking.Native.NativeRoomController.ApplyRemoteAudioPreference(string, bool)
at NativeRoomController.cs:477
Convai.Runtime.Adapters.Networking.ConvaiRoomManager.SetRemoteAudioEnabled(string, bool)
at ConvaiRoomManager.cs:906
Convai.Runtime.Components.ConvaiCharacter.SetRemoteAudioEnabled(bool)
at ConvaiCharacter.cs:290
Cause
The livekit_ffi.dll auto-downloaded by the SDK (0.12.43) does not export the EnableRemoteTrackRequest entry point required by the C# bindings. Manually replacing with 0.12.47 stops the crash but the fundamental problem remains: _enableRemoteAudio cannot be toggled at runtime without crashing, making it impossible to mute/unmute individual characters via the SDKβs own audio API.
Workaround applied
Set _enableRemoteAudio = false on all characters. This prevents the crash but means audio routing cannot use the intended SDK mechanism.
Bug 2 β SceneCharacterDiscovery Ignores MonoBehaviour.enabled; Requires GameObject.SetActive
Symptom
Setting ConvaiCharacter.enabled = false on a component does not prevent it from being discovered and registered with the room on ConnectAsync. Even with all but one ConvaiCharacter disabled, the room connects using a different (wrong) character.
Cause
SceneCharacterDiscovery internally calls:
Object.FindObjectsByType(implementerType, FindObjectsInactive.Exclude, FindObjectsSortMode.None)
FindObjectsInactive.Exclude filters by GameObject.activeInHierarchy, not by MonoBehaviour.enabled. A disabled component on an active GameObject is still returned and therefore still registered with the room.
Question for SDK team
Is this by design? If character registration is intended to be controlled at runtime, should SceneCharacterDiscovery respect MonoBehaviour.enabled, or is there a supported API to selectively register/deregister individual ConvaiCharacter components without toggling the entire GameObject?
Workaround applied
Before calling ConnectAsync, temporarily call SetActive(false) on all character GameObjects except the selected one, then restore them after ConnectAsync completes. This is fragile and interferes with other scene systems (Animators, AudioSources, etc. are interrupted during the active/inactive cycle).
Bug 3 β ConnectAsync Fails When Called While a Previous DisconnectAsync Is In-Flight
Symptom
[ConvaiRoomManager] ConnectAsync called while disconnecting.
[NativeTransport] Disconnect timed out waiting for Disconnected event.
ConnectAsync returns false. The room is left in a broken state β not connected, not disconnected.
Cause
DisconnectAsync is asynchronous and can take up to ~5 seconds to complete internally (it fires a timeout). If ConnectAsync is called during this window β even after DisconnectAsyncβs returned Task has completed β the roomβs internal state machine is still in a Disconnecting state and rejects the connect.
There is no public API to query whether the room is in an intermediate state. ConvaiManager.Instance.IsConnected returns false once the task completes, but the internal transport is still cleaning up.
Repro steps
-
Enter conversation with Character A β room connects
-
Press back quickly β
DisconnectAsyncstarts -
Immediately select Character B β
ConnectAsynccalled while disconnect is still running internally -
ConnectAsyncreturnsfalse; room is broken for the remainder of the session
Question for SDK team
Is there a supported way to await full transport cleanup before calling ConnectAsync? Is there an event or state property that signals the room is fully idle (not just IsConnected == false)?
Workaround applied
After DisconnectAsync completes, poll IsConnected frame-by-frame for up to 6 seconds before calling ConnectAsync. This is unreliable and adds significant latency to every character switch.
What We Are Trying to Achieve
A runtime character switcher where:
-
The player selects one of N characters from a UI
-
Only that character joins the Convai room and responds to voice
-
The player can return to the selection screen and pick a different character
-
The cycle repeats cleanly for the session
This is a straightforward multi-NPC scene. The SDK appears to be designed around a single-character-per-scene assumption. Confirmation of the correct SDK pattern for multi-character scenes with runtime switching would resolve all three issues.
Inspector Settings on All ConvaiCharacter Components
Property
Value
Notes
_autoConnect
false
We call StartConversationAsync manually
_enableRemoteAudio
false
Forced off β crashes if true (Bug 1)
_enableSessionResume
false
Set off β SDK was rejoining cached character instead of discovered one
Our Custom Scripts
Three scripts were written to work around the above bugs. Included below in full.
ConvaiNPCActivator.cs (new file)
Manages room connect/disconnect and character isolation for multi-NPC switching.
using Convai.Runtime.Components;
using System.Collections;
using UnityEngine;
// REWRITTEN: component.enabled does not filter FindObjectsByType β only SetActive does.
namespace Propheti
{
[DefaultExecutionOrder(-5)]
public class ConvaiNPCActivator : MonoBehaviour
{
[Header("Scene References")]
[Tooltip("Same Transform as CharacterSwitcher (Carousel_Stage). " +
"Child order must match the NPCRotationUI card order.")]
[SerializeField] private Transform showcaseRoot;
private ConvaiCharacter[] _characters;
private int _activeIndex = -1;
private Coroutine _activeRoutine;
private const string LogPrefix = "[ConvaiNPCActivator]";
private void Start()
{
if (showcaseRoot == null)
{
Debug.LogError($"{LogPrefix} showcaseRoot not assigned.", this);
enabled = false;
return;
}
CacheCharacters();
}
public void ActivateNPC(int index)
{
if (_characters == null || index < 0 || index >= _characters.Length || _characters[index] == null)
{
Debug.LogError($"{LogPrefix} ActivateNPC: invalid index {index}.", this);
return;
}
_activeIndex = index;
if (_activeRoutine != null) StopCoroutine(_activeRoutine);
_activeRoutine = StartCoroutine(SwitchCharacterRoutine(index));
}
public void DeactivateAll()
{
_activeIndex = -1;
if (_activeRoutine != null) StopCoroutine(_activeRoutine);
_activeRoutine = StartCoroutine(DisconnectRoutine());
}
private void CacheCharacters()
{
_characters = new ConvaiCharacter[showcaseRoot.childCount];
for (int i = 0; i < showcaseRoot.childCount; i++)
{
Transform child = showcaseRoot.GetChild(i);
ConvaiCharacter c = child.GetComponentInChildren<ConvaiCharacter>(includeInactive: true);
if (c == null)
Debug.LogWarning($"{LogPrefix} No ConvaiCharacter on child [{i}] '{child.name}'.", this);
else
Debug.Log($"{LogPrefix} Cached [{i}] '{c.CharacterName}' id={c.CharacterId}");
_characters[i] = c;
}
}
private IEnumerator SwitchCharacterRoutine(int index)
{
// Workaround for Bug 2: must SetActive(false) on all other GOs β
// disabling the ConvaiCharacter component alone does not work.
for (int i = 0; i < _characters.Length; i++)
if (_characters[i] != null)
_characters[i].gameObject.SetActive(i == index);
Debug.Log($"{LogPrefix} Isolated '{_characters[index].CharacterName}' for room reconnect.");
// Workaround for Bug 3: call DisconnectAsync then poll IsConnected
// because the task completing does not mean the transport is idle.
if (ConvaiManager.Instance != null && ConvaiManager.Instance.IsConnected)
{
var disconnectTask = ConvaiManager.Instance.DisconnectAsync();
yield return new WaitUntil(() => disconnectTask.IsCompleted);
Debug.Log($"{LogPrefix} DisconnectAsync completed.");
}
const float disconnectTimeout = 6f;
float elapsed = 0f;
while (ConvaiManager.Instance != null && ConvaiManager.Instance.IsConnected && elapsed < disconnectTimeout)
{
elapsed += Time.deltaTime;
yield return null;
}
if (ConvaiManager.Instance != null && ConvaiManager.Instance.IsConnected)
{
Debug.LogError($"{LogPrefix} Room failed to disconnect after {disconnectTimeout}s. Aborting.");
RestoreAllActive();
yield break;
}
Debug.Log($"{LogPrefix} Room fully disconnected. Reconnecting for '{_characters[index].CharacterName}'.");
if (ConvaiManager.Instance == null)
{
Debug.LogError($"{LogPrefix} ConvaiManager.Instance is null.");
RestoreAllActive();
yield break;
}
var connectTask = ConvaiManager.Instance.ConnectAsync();
yield return new WaitUntil(() => connectTask.IsCompleted);
RestoreAllActive();
if (!connectTask.Result)
{
Debug.LogError($"{LogPrefix} ConnectAsync failed for '{_characters[index].CharacterName}'.");
yield break;
}
Debug.Log($"{LogPrefix} Room reconnected for '{_characters[index].CharacterName}'.");
var conversationTask = _characters[index].StartConversationAsync();
yield return new WaitUntil(() => conversationTask.IsCompleted);
if (!conversationTask.Result)
Debug.LogWarning($"{LogPrefix} StartConversationAsync returned false for '{_characters[index].CharacterName}'.");
else
Debug.Log($"{LogPrefix} Conversation ready: '{_characters[index].CharacterName}'.");
}
private IEnumerator DisconnectRoutine()
{
if (ConvaiManager.Instance == null || !ConvaiManager.Instance.IsConnected) yield break;
var task = ConvaiManager.Instance.DisconnectAsync();
yield return new WaitUntil(() => task.IsCompleted);
Debug.Log($"{LogPrefix} Room disconnected on DeactivateAll.");
}
private void RestoreAllActive()
{
foreach (ConvaiCharacter c in _characters)
if (c != null) c.gameObject.SetActive(true);
}
}
}
CharacterSwitcher.cs (modified β SDK references removed)
Controls visual carousel and camera only. All Convai SDK calls removed.
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.Events;
using Convai.ShowcaseCamera;
public class CharacterSwitcher : MonoBehaviour
{
[Header("References")]
public Transform showcaseRoot;
[Header("Default Character")]
public string defaultCharacterName = "Convai Character Daniel";
[Header("Camera")]
public Camera showcaseCamera;
[Range(2f, 10f)] public float cameraDistance = 4.5f;
public Vector3 cameraLookOffset = new Vector3(0f, 1.6f, 0f);
[Header("Transition")]
public float transitionDuration = 0.3f;
[Header("UI Buttons (optional)")]
public List<UnityEngine.UI.Button> characterButtons;
public UnityEvent<int> onCharacterChanged = new UnityEvent<int>();
public bool IsTransitioning { get; private set; }
public Transform CurrentCharacterTransform
{
get
{
if (showcaseRoot == null) return null;
if (_currentIndex < 0 || _currentIndex >= showcaseRoot.childCount) return null;
return showcaseRoot.GetChild(_currentIndex);
}
}
public int CurrentIndex => _currentIndex;
private Dictionary<string, int> _characterMap = new Dictionary<string, int>();
private int _currentIndex = -1;
private Coroutine _transitionCoroutine;
private const string LOG = "[CharacterSwitcher]";
private void Awake()
{
BuildCharacterMap();
if (showcaseCamera == null)
showcaseCamera = Camera.main;
}
private void Start()
{
StartCoroutine(InitialiseAfterSDK());
}
private void BuildCharacterMap()
{
_characterMap.Clear();
if (showcaseRoot == null) { Debug.LogError($"{LOG} showcaseRoot not assigned!"); return; }
for (int i = 0; i < showcaseRoot.childCount; i++)
{
string n = showcaseRoot.GetChild(i).name.Trim();
if (_characterMap.ContainsKey(n)) Debug.LogWarning($"{LOG} Duplicate name '{n}' at index {i}.");
else _characterMap[n] = i;
}
}
private IEnumerator InitialiseAfterSDK()
{
yield return null;
for (int i = 0; i < showcaseRoot.childCount; i++)
SetRenderersEnabled(showcaseRoot.GetChild(i).gameObject, false);
SelectDefaultCharacterByName();
RegisterButtonListeners();
}
private void RegisterButtonListeners()
{
if (characterButtons == null) return;
for (int i = 0; i < characterButtons.Count; i++)
{
if (characterButtons[i] == null) continue;
int idx = i;
characterButtons[i].onClick.AddListener(() => HandleCharacterSelection(idx));
}
}
public void Next() { if (showcaseRoot != null && !IsTransitioning) HandleCharacterSelection((_currentIndex + 1) % showcaseRoot.childCount); }
public void Previous() { if (showcaseRoot != null && !IsTransitioning) HandleCharacterSelection((_currentIndex - 1 + showcaseRoot.childCount) % showcaseRoot.childCount); }
public void GoToIndex(int index) => HandleCharacterSelection(index);
public void EnterSelectionMode() { IsTransitioning = false; Debug.Log($"{LOG} EnterSelectionMode()"); }
public void EnterConversationMode() => Debug.Log($"{LOG} EnterConversationMode()");
public void AimCameraAtCharacter(int index)
{
if (showcaseRoot == null || showcaseCamera == null || index < 0 || index >= showcaseRoot.childCount) return;
Transform t = showcaseRoot.GetChild(index);
Vector3 lookTarget = t.position + cameraLookOffset;
showcaseCamera.transform.position = lookTarget - (t.forward * cameraDistance);
showcaseCamera.transform.LookAt(lookTarget);
}
private void SelectDefaultCharacterByName()
{
string trimmed = defaultCharacterName.Trim();
for (int i = 0; i < showcaseRoot.childCount; i++)
if (showcaseRoot.GetChild(i).name.Trim().Equals(trimmed, System.StringComparison.OrdinalIgnoreCase))
{ HandleCharacterSelection(i); return; }
HandleCharacterSelection(0);
}
private void HandleCharacterSelection(int index)
{
if (showcaseRoot == null || index < 0 || index >= showcaseRoot.childCount || index == _currentIndex) return;
if (_transitionCoroutine != null) StopCoroutine(_transitionCoroutine);
_transitionCoroutine = StartCoroutine(TransitionTo(index));
}
private IEnumerator TransitionTo(int newIndex)
{
IsTransitioning = true;
if (_currentIndex >= 0 && _currentIndex < showcaseRoot.childCount)
SetRenderersEnabled(showcaseRoot.GetChild(_currentIndex).gameObject, false);
if (transitionDuration > 0f) yield return new WaitForSeconds(transitionDuration * 0.5f);
_currentIndex = newIndex;
SetRenderersEnabled(showcaseRoot.GetChild(_currentIndex).gameObject, true);
if (transitionDuration > 0f) yield return new WaitForSeconds(transitionDuration * 0.5f);
IsTransitioning = false;
_transitionCoroutine = null;
onCharacterChanged.Invoke(_currentIndex);
Debug.Log($"{LOG} β
Active: [{_currentIndex}] '{showcaseRoot.GetChild(_currentIndex).name}'");
}
private void SetRenderersEnabled(GameObject character, bool enabled)
{
if (character == null) return;
foreach (var r in character.GetComponentsInChildren<Renderer>(true))
r.enabled = enabled;
}
// Audio routing delegated entirely to ConvaiNPCActivator.
private void NotifyConvaiActivate(GameObject character) { }
private void NotifyConvaiDeactivate(GameObject character) { }
}
NPCSelectionScreenManager.cs (modified β UI flow controller)
Orchestrates panel transitions and calls ConvaiNPCActivator for SDK routing.
using System.Collections;
using UnityEngine;
using UnityEngine.UI;
namespace Propheti
{
[DefaultExecutionOrder(-10)]
public class NPCSelectionScreenManager : MonoBehaviour
{
[Header("Panels")]
[SerializeField] private CanvasGroup selectionPanel;
[SerializeField] private CanvasGroup conversationHUD;
[Header("References")]
[SerializeField] private NPCRotationUI rotationUI;
[SerializeField] private CharacterSwitcher characterSwitcher;
[SerializeField] private ConvaiNPCActivator convaiActivator;
[Header("Transition")]
[SerializeField] private float fadeDuration = 0.35f;
private bool _inConversation;
private Coroutine _hudFade;
private Coroutine _panelFade;
private const string LogPrefix = "[NPCSelectionScreenManager]";
private void Awake()
{
if (characterSwitcher == null)
characterSwitcher = FindAnyObjectByType<CharacterSwitcher>();
DisableConvaiChatPanelBackground();
if (!ValidateReferences()) return;
rotationUI.OnCharacterConfirmed += HandleCharacterConfirmed;
}
private void Start()
{
ShowSelectionPanel(instant: true);
HideConversationHUD(instant: true);
}
private void OnDestroy()
{
if (rotationUI != null)
rotationUI.OnCharacterConfirmed -= HandleCharacterConfirmed;
}
public void ReturnToSelection()
{
if (!_inConversation) return;
_inConversation = false;
convaiActivator.DeactivateAll();
characterSwitcher.EnterSelectionMode();
HideConversationHUD(instant: false);
ShowSelectionPanel(instant: false);
}
private void HandleCharacterConfirmed(int index)
{
if (_inConversation) return;
_inConversation = true;
IOSAudioSession.Configure();
characterSwitcher.EnterConversationMode();
characterSwitcher.GoToIndex(index);
characterSwitcher.AimCameraAtCharacter(index);
convaiActivator.ActivateNPC(index);
HideSelectionPanel(instant: false);
ShowConversationHUD(instant: false);
}
private void ShowSelectionPanel(bool instant)
{
if (selectionPanel == null) return;
if (_panelFade != null) StopCoroutine(_panelFade);
selectionPanel.gameObject.SetActive(true);
if (instant) { selectionPanel.alpha = 1f; selectionPanel.interactable = true; selectionPanel.blocksRaycasts = true; }
else _panelFade = StartCoroutine(FadeCanvasGroup(selectionPanel, 0f, 1f, fadeDuration, setInteractable: true));
}
private void HideSelectionPanel(bool instant)
{
if (selectionPanel == null) return;
if (_panelFade != null) StopCoroutine(_panelFade);
if (instant) { selectionPanel.alpha = 0f; selectionPanel.interactable = false; selectionPanel.blocksRaycasts = false; selectionPanel.gameObject.SetActive(false); }
else _panelFade = StartCoroutine(FadeCanvasGroup(selectionPanel, 1f, 0f, fadeDuration, setInteractable: false, deactivateOnComplete: true));
}
private void ShowConversationHUD(bool instant)
{
if (conversationHUD == null) return;
if (_hudFade != null) StopCoroutine(_hudFade);
conversationHUD.gameObject.SetActive(true);
if (instant) { conversationHUD.alpha = 1f; conversationHUD.interactable = true; conversationHUD.blocksRaycasts = true; }
else _hudFade = StartCoroutine(FadeCanvasGroup(conversationHUD, 0f, 1f, fadeDuration, setInteractable: true));
}
private void HideConversationHUD(bool instant)
{
if (conversationHUD == null) return;
if (_hudFade != null) StopCoroutine(_hudFade);
if (instant) { conversationHUD.alpha = 0f; conversationHUD.interactable = false; conversationHUD.blocksRaycasts = false; conversationHUD.gameObject.SetActive(false); }
else _hudFade = StartCoroutine(FadeCanvasGroup(conversationHUD, 1f, 0f, fadeDuration, setInteractable: false, deactivateOnComplete: true));
}
private static IEnumerator FadeCanvasGroup(CanvasGroup cg, float fromAlpha, float toAlpha, float duration, bool setInteractable, bool deactivateOnComplete = false)
{
cg.alpha = fromAlpha; cg.interactable = false; cg.blocksRaycasts = false;
float elapsed = 0f;
while (elapsed < duration) { elapsed += Time.deltaTime; cg.alpha = Mathf.Lerp(fromAlpha, toAlpha, Mathf.Clamp01(elapsed / duration)); yield return null; }
cg.alpha = toAlpha;
if (setInteractable) { cg.interactable = true; cg.blocksRaycasts = true; }
if (deactivateOnComplete && toAlpha <= 0f) cg.gameObject.SetActive(false);
}
private bool ValidateReferences()
{
bool ok = true;
if (selectionPanel == null) { Debug.LogError($"{LogPrefix} selectionPanel not assigned.", this); ok = false; }
if (rotationUI == null) { Debug.LogError($"{LogPrefix} rotationUI not assigned.", this); ok = false; }
if (characterSwitcher == null) { Debug.LogError($"{LogPrefix} characterSwitcher not assigned.", this); ok = false; }
if (convaiActivator == null) { Debug.LogError($"{LogPrefix} convaiActivator not assigned.", this); ok = false; }
if (!ok) enabled = false;
return ok;
}
private void DisableConvaiChatPanelBackground()
{
GameObject canvas = GameObject.Find("Convai Transcript Canvas - Mobile Chat");
if (canvas == null) return;
Transform panel = canvas.transform.Find("Panel");
if (panel == null) return;
Image bg = panel.GetComponent<Image>();
if (bg != null) bg.enabled = false;
}
}
}