Does the Convai SDK support runtime character switching without crashes

Environment

Unity

6000.4

Convai SDK

4.0.0 (com.convai.convai-sdk-for-unity)

Render Pipeline

URP 17.4.0

Platform

Windows x86-64

LiveKit FFI DLL

Manually replaced with livekit_ffi 0.12.47 (auto-downloader fetched incompatible 0.12.43)


Problem Summary

A scene has 5 ConvaiCharacter components on separate GameObjects under one parent (Carousel_Stage). The player selects one character at a time from a UI carousel. The expected behaviour is that only the selected character responds to voice input. There are three compounding bugs blocking this.


Bug 1 β€” EnableRemoteTrackRequest Missing from Native FFI DLL

Symptom

Every time a ConvaiCharacter with _enableRemoteAudio = true joins the room, this exception fires:

Exception: Unknown request type: LiveKit.Proto.EnableRemoteTrackRequest
LiveKit.Internal.FFIClients.Requests.FfiRequestWrap`1[T].Send()
  at FfiRequestWrap.cs:57
LiveKit.IRemoteTrack.SetEnabled(bool enabled)
  at Track.cs:59
Convai.Infrastructure.Networking.Native.NativeRemoteAudioTrack.SetRemoteAudioEnabled(bool)
  at NativeRemoteAudioTrack.cs:28
Convai.Infrastructure.Networking.Native.NativeRoomController.ApplyRemoteAudioPreference(string, bool)
  at NativeRoomController.cs:477
Convai.Runtime.Adapters.Networking.ConvaiRoomManager.SetRemoteAudioEnabled(string, bool)
  at ConvaiRoomManager.cs:906
Convai.Runtime.Components.ConvaiCharacter.SetRemoteAudioEnabled(bool)
  at ConvaiCharacter.cs:290

Cause

The livekit_ffi.dll auto-downloaded by the SDK (0.12.43) does not export the EnableRemoteTrackRequest entry point required by the C# bindings. Manually replacing with 0.12.47 stops the crash but the fundamental problem remains: _enableRemoteAudio cannot be toggled at runtime without crashing, making it impossible to mute/unmute individual characters via the SDK’s own audio API.

Workaround applied

Set _enableRemoteAudio = false on all characters. This prevents the crash but means audio routing cannot use the intended SDK mechanism.


Bug 2 β€” SceneCharacterDiscovery Ignores MonoBehaviour.enabled; Requires GameObject.SetActive

Symptom

Setting ConvaiCharacter.enabled = false on a component does not prevent it from being discovered and registered with the room on ConnectAsync. Even with all but one ConvaiCharacter disabled, the room connects using a different (wrong) character.

Cause

SceneCharacterDiscovery internally calls:

Object.FindObjectsByType(implementerType, FindObjectsInactive.Exclude, FindObjectsSortMode.None)

FindObjectsInactive.Exclude filters by GameObject.activeInHierarchy, not by MonoBehaviour.enabled. A disabled component on an active GameObject is still returned and therefore still registered with the room.

Question for SDK team

Is this by design? If character registration is intended to be controlled at runtime, should SceneCharacterDiscovery respect MonoBehaviour.enabled, or is there a supported API to selectively register/deregister individual ConvaiCharacter components without toggling the entire GameObject?

Workaround applied

Before calling ConnectAsync, temporarily call SetActive(false) on all character GameObjects except the selected one, then restore them after ConnectAsync completes. This is fragile and interferes with other scene systems (Animators, AudioSources, etc. are interrupted during the active/inactive cycle).


Bug 3 β€” ConnectAsync Fails When Called While a Previous DisconnectAsync Is In-Flight

Symptom

[ConvaiRoomManager] ConnectAsync called while disconnecting.
[NativeTransport] Disconnect timed out waiting for Disconnected event.

ConnectAsync returns false. The room is left in a broken state β€” not connected, not disconnected.

Cause

DisconnectAsync is asynchronous and can take up to ~5 seconds to complete internally (it fires a timeout). If ConnectAsync is called during this window β€” even after DisconnectAsync’s returned Task has completed β€” the room’s internal state machine is still in a Disconnecting state and rejects the connect.

There is no public API to query whether the room is in an intermediate state. ConvaiManager.Instance.IsConnected returns false once the task completes, but the internal transport is still cleaning up.

Repro steps

  1. Enter conversation with Character A β†’ room connects

  2. Press back quickly β†’ DisconnectAsync starts

  3. Immediately select Character B β†’ ConnectAsync called while disconnect is still running internally

  4. ConnectAsync returns false; room is broken for the remainder of the session

Question for SDK team

Is there a supported way to await full transport cleanup before calling ConnectAsync? Is there an event or state property that signals the room is fully idle (not just IsConnected == false)?

Workaround applied

After DisconnectAsync completes, poll IsConnected frame-by-frame for up to 6 seconds before calling ConnectAsync. This is unreliable and adds significant latency to every character switch.


What We Are Trying to Achieve

A runtime character switcher where:

  1. The player selects one of N characters from a UI

  2. Only that character joins the Convai room and responds to voice

  3. The player can return to the selection screen and pick a different character

  4. The cycle repeats cleanly for the session

This is a straightforward multi-NPC scene. The SDK appears to be designed around a single-character-per-scene assumption. Confirmation of the correct SDK pattern for multi-character scenes with runtime switching would resolve all three issues.


Inspector Settings on All ConvaiCharacter Components

Property

Value

Notes

_autoConnect

false

We call StartConversationAsync manually

_enableRemoteAudio

false

Forced off β€” crashes if true (Bug 1)

_enableSessionResume

false

Set off β€” SDK was rejoining cached character instead of discovered one


Our Custom Scripts

Three scripts were written to work around the above bugs. Included below in full.


ConvaiNPCActivator.cs (new file)

Manages room connect/disconnect and character isolation for multi-NPC switching.

using Convai.Runtime.Components;
using System.Collections;
using UnityEngine;

// REWRITTEN: component.enabled does not filter FindObjectsByType β€” only SetActive does.
namespace Propheti
{
    [DefaultExecutionOrder(-5)]
    public class ConvaiNPCActivator : MonoBehaviour
    {
        [Header("Scene References")]
        [Tooltip("Same Transform as CharacterSwitcher (Carousel_Stage). " +
                 "Child order must match the NPCRotationUI card order.")]
        [SerializeField] private Transform showcaseRoot;

        private ConvaiCharacter[] _characters;
        private int _activeIndex = -1;
        private Coroutine _activeRoutine;
        private const string LogPrefix = "[ConvaiNPCActivator]";

        private void Start()
        {
            if (showcaseRoot == null)
            {
                Debug.LogError($"{LogPrefix} showcaseRoot not assigned.", this);
                enabled = false;
                return;
            }
            CacheCharacters();
        }

        public void ActivateNPC(int index)
        {
            if (_characters == null || index < 0 || index >= _characters.Length || _characters[index] == null)
            {
                Debug.LogError($"{LogPrefix} ActivateNPC: invalid index {index}.", this);
                return;
            }
            _activeIndex = index;
            if (_activeRoutine != null) StopCoroutine(_activeRoutine);
            _activeRoutine = StartCoroutine(SwitchCharacterRoutine(index));
        }

        public void DeactivateAll()
        {
            _activeIndex = -1;
            if (_activeRoutine != null) StopCoroutine(_activeRoutine);
            _activeRoutine = StartCoroutine(DisconnectRoutine());
        }

        private void CacheCharacters()
        {
            _characters = new ConvaiCharacter[showcaseRoot.childCount];
            for (int i = 0; i < showcaseRoot.childCount; i++)
            {
                Transform child = showcaseRoot.GetChild(i);
                ConvaiCharacter c = child.GetComponentInChildren<ConvaiCharacter>(includeInactive: true);
                if (c == null)
                    Debug.LogWarning($"{LogPrefix} No ConvaiCharacter on child [{i}] '{child.name}'.", this);
                else
                    Debug.Log($"{LogPrefix} Cached [{i}] '{c.CharacterName}' id={c.CharacterId}");
                _characters[i] = c;
            }
        }

        private IEnumerator SwitchCharacterRoutine(int index)
        {
            // Workaround for Bug 2: must SetActive(false) on all other GOs β€”
            // disabling the ConvaiCharacter component alone does not work.
            for (int i = 0; i < _characters.Length; i++)
                if (_characters[i] != null)
                    _characters[i].gameObject.SetActive(i == index);

            Debug.Log($"{LogPrefix} Isolated '{_characters[index].CharacterName}' for room reconnect.");

            // Workaround for Bug 3: call DisconnectAsync then poll IsConnected
            // because the task completing does not mean the transport is idle.
            if (ConvaiManager.Instance != null && ConvaiManager.Instance.IsConnected)
            {
                var disconnectTask = ConvaiManager.Instance.DisconnectAsync();
                yield return new WaitUntil(() => disconnectTask.IsCompleted);
                Debug.Log($"{LogPrefix} DisconnectAsync completed.");
            }

            const float disconnectTimeout = 6f;
            float elapsed = 0f;
            while (ConvaiManager.Instance != null && ConvaiManager.Instance.IsConnected && elapsed < disconnectTimeout)
            {
                elapsed += Time.deltaTime;
                yield return null;
            }

            if (ConvaiManager.Instance != null && ConvaiManager.Instance.IsConnected)
            {
                Debug.LogError($"{LogPrefix} Room failed to disconnect after {disconnectTimeout}s. Aborting.");
                RestoreAllActive();
                yield break;
            }

            Debug.Log($"{LogPrefix} Room fully disconnected. Reconnecting for '{_characters[index].CharacterName}'.");

            if (ConvaiManager.Instance == null)
            {
                Debug.LogError($"{LogPrefix} ConvaiManager.Instance is null.");
                RestoreAllActive();
                yield break;
            }

            var connectTask = ConvaiManager.Instance.ConnectAsync();
            yield return new WaitUntil(() => connectTask.IsCompleted);

            RestoreAllActive();

            if (!connectTask.Result)
            {
                Debug.LogError($"{LogPrefix} ConnectAsync failed for '{_characters[index].CharacterName}'.");
                yield break;
            }

            Debug.Log($"{LogPrefix} Room reconnected for '{_characters[index].CharacterName}'.");

            var conversationTask = _characters[index].StartConversationAsync();
            yield return new WaitUntil(() => conversationTask.IsCompleted);

            if (!conversationTask.Result)
                Debug.LogWarning($"{LogPrefix} StartConversationAsync returned false for '{_characters[index].CharacterName}'.");
            else
                Debug.Log($"{LogPrefix} Conversation ready: '{_characters[index].CharacterName}'.");
        }

        private IEnumerator DisconnectRoutine()
        {
            if (ConvaiManager.Instance == null || !ConvaiManager.Instance.IsConnected) yield break;
            var task = ConvaiManager.Instance.DisconnectAsync();
            yield return new WaitUntil(() => task.IsCompleted);
            Debug.Log($"{LogPrefix} Room disconnected on DeactivateAll.");
        }

        private void RestoreAllActive()
        {
            foreach (ConvaiCharacter c in _characters)
                if (c != null) c.gameObject.SetActive(true);
        }
    }
}

CharacterSwitcher.cs (modified β€” SDK references removed)

Controls visual carousel and camera only. All Convai SDK calls removed.

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.Events;
using Convai.ShowcaseCamera;

public class CharacterSwitcher : MonoBehaviour
{
    [Header("References")]
    public Transform showcaseRoot;

    [Header("Default Character")]
    public string defaultCharacterName = "Convai Character Daniel";

    [Header("Camera")]
    public Camera showcaseCamera;
    [Range(2f, 10f)] public float cameraDistance = 4.5f;
    public Vector3 cameraLookOffset = new Vector3(0f, 1.6f, 0f);

    [Header("Transition")]
    public float transitionDuration = 0.3f;

    [Header("UI Buttons (optional)")]
    public List<UnityEngine.UI.Button> characterButtons;

    public UnityEvent<int> onCharacterChanged = new UnityEvent<int>();
    public bool IsTransitioning { get; private set; }

    public Transform CurrentCharacterTransform
    {
        get
        {
            if (showcaseRoot == null) return null;
            if (_currentIndex < 0 || _currentIndex >= showcaseRoot.childCount) return null;
            return showcaseRoot.GetChild(_currentIndex);
        }
    }

    public int CurrentIndex => _currentIndex;

    private Dictionary<string, int> _characterMap = new Dictionary<string, int>();
    private int _currentIndex = -1;
    private Coroutine _transitionCoroutine;
    private const string LOG = "[CharacterSwitcher]";

    private void Awake()
    {
        BuildCharacterMap();
        if (showcaseCamera == null)
            showcaseCamera = Camera.main;
    }

    private void Start()
    {
        StartCoroutine(InitialiseAfterSDK());
    }

    private void BuildCharacterMap()
    {
        _characterMap.Clear();
        if (showcaseRoot == null) { Debug.LogError($"{LOG} showcaseRoot not assigned!"); return; }
        for (int i = 0; i < showcaseRoot.childCount; i++)
        {
            string n = showcaseRoot.GetChild(i).name.Trim();
            if (_characterMap.ContainsKey(n)) Debug.LogWarning($"{LOG} Duplicate name '{n}' at index {i}.");
            else _characterMap[n] = i;
        }
    }

    private IEnumerator InitialiseAfterSDK()
    {
        yield return null;
        for (int i = 0; i < showcaseRoot.childCount; i++)
            SetRenderersEnabled(showcaseRoot.GetChild(i).gameObject, false);
        SelectDefaultCharacterByName();
        RegisterButtonListeners();
    }

    private void RegisterButtonListeners()
    {
        if (characterButtons == null) return;
        for (int i = 0; i < characterButtons.Count; i++)
        {
            if (characterButtons[i] == null) continue;
            int idx = i;
            characterButtons[i].onClick.AddListener(() => HandleCharacterSelection(idx));
        }
    }

    public void Next() { if (showcaseRoot != null && !IsTransitioning) HandleCharacterSelection((_currentIndex + 1) % showcaseRoot.childCount); }
    public void Previous() { if (showcaseRoot != null && !IsTransitioning) HandleCharacterSelection((_currentIndex - 1 + showcaseRoot.childCount) % showcaseRoot.childCount); }
    public void GoToIndex(int index) => HandleCharacterSelection(index);
    public void EnterSelectionMode() { IsTransitioning = false; Debug.Log($"{LOG} EnterSelectionMode()"); }
    public void EnterConversationMode() => Debug.Log($"{LOG} EnterConversationMode()");
    public void AimCameraAtCharacter(int index)
    {
        if (showcaseRoot == null || showcaseCamera == null || index < 0 || index >= showcaseRoot.childCount) return;
        Transform t = showcaseRoot.GetChild(index);
        Vector3 lookTarget = t.position + cameraLookOffset;
        showcaseCamera.transform.position = lookTarget - (t.forward * cameraDistance);
        showcaseCamera.transform.LookAt(lookTarget);
    }

    private void SelectDefaultCharacterByName()
    {
        string trimmed = defaultCharacterName.Trim();
        for (int i = 0; i < showcaseRoot.childCount; i++)
            if (showcaseRoot.GetChild(i).name.Trim().Equals(trimmed, System.StringComparison.OrdinalIgnoreCase))
                { HandleCharacterSelection(i); return; }
        HandleCharacterSelection(0);
    }

    private void HandleCharacterSelection(int index)
    {
        if (showcaseRoot == null || index < 0 || index >= showcaseRoot.childCount || index == _currentIndex) return;
        if (_transitionCoroutine != null) StopCoroutine(_transitionCoroutine);
        _transitionCoroutine = StartCoroutine(TransitionTo(index));
    }

    private IEnumerator TransitionTo(int newIndex)
    {
        IsTransitioning = true;
        if (_currentIndex >= 0 && _currentIndex < showcaseRoot.childCount)
            SetRenderersEnabled(showcaseRoot.GetChild(_currentIndex).gameObject, false);
        if (transitionDuration > 0f) yield return new WaitForSeconds(transitionDuration * 0.5f);
        _currentIndex = newIndex;
        SetRenderersEnabled(showcaseRoot.GetChild(_currentIndex).gameObject, true);
        if (transitionDuration > 0f) yield return new WaitForSeconds(transitionDuration * 0.5f);
        IsTransitioning = false;
        _transitionCoroutine = null;
        onCharacterChanged.Invoke(_currentIndex);
        Debug.Log($"{LOG} βœ… Active: [{_currentIndex}] '{showcaseRoot.GetChild(_currentIndex).name}'");
    }

    private void SetRenderersEnabled(GameObject character, bool enabled)
    {
        if (character == null) return;
        foreach (var r in character.GetComponentsInChildren<Renderer>(true))
            r.enabled = enabled;
    }

    // Audio routing delegated entirely to ConvaiNPCActivator.
    private void NotifyConvaiActivate(GameObject character) { }
    private void NotifyConvaiDeactivate(GameObject character) { }
}

NPCSelectionScreenManager.cs (modified β€” UI flow controller)

Orchestrates panel transitions and calls ConvaiNPCActivator for SDK routing.

using System.Collections;
using UnityEngine;
using UnityEngine.UI;

namespace Propheti
{
    [DefaultExecutionOrder(-10)]
    public class NPCSelectionScreenManager : MonoBehaviour
    {
        [Header("Panels")]
        [SerializeField] private CanvasGroup selectionPanel;
        [SerializeField] private CanvasGroup conversationHUD;

        [Header("References")]
        [SerializeField] private NPCRotationUI rotationUI;
        [SerializeField] private CharacterSwitcher characterSwitcher;
        [SerializeField] private ConvaiNPCActivator convaiActivator;

        [Header("Transition")]
        [SerializeField] private float fadeDuration = 0.35f;

        private bool _inConversation;
        private Coroutine _hudFade;
        private Coroutine _panelFade;
        private const string LogPrefix = "[NPCSelectionScreenManager]";

        private void Awake()
        {
            if (characterSwitcher == null)
                characterSwitcher = FindAnyObjectByType<CharacterSwitcher>();
            DisableConvaiChatPanelBackground();
            if (!ValidateReferences()) return;
            rotationUI.OnCharacterConfirmed += HandleCharacterConfirmed;
        }

        private void Start()
        {
            ShowSelectionPanel(instant: true);
            HideConversationHUD(instant: true);
        }

        private void OnDestroy()
        {
            if (rotationUI != null)
                rotationUI.OnCharacterConfirmed -= HandleCharacterConfirmed;
        }

        public void ReturnToSelection()
        {
            if (!_inConversation) return;
            _inConversation = false;
            convaiActivator.DeactivateAll();
            characterSwitcher.EnterSelectionMode();
            HideConversationHUD(instant: false);
            ShowSelectionPanel(instant: false);
        }

        private void HandleCharacterConfirmed(int index)
        {
            if (_inConversation) return;
            _inConversation = true;
            IOSAudioSession.Configure();
            characterSwitcher.EnterConversationMode();
            characterSwitcher.GoToIndex(index);
            characterSwitcher.AimCameraAtCharacter(index);
            convaiActivator.ActivateNPC(index);
            HideSelectionPanel(instant: false);
            ShowConversationHUD(instant: false);
        }

        private void ShowSelectionPanel(bool instant)
        {
            if (selectionPanel == null) return;
            if (_panelFade != null) StopCoroutine(_panelFade);
            selectionPanel.gameObject.SetActive(true);
            if (instant) { selectionPanel.alpha = 1f; selectionPanel.interactable = true; selectionPanel.blocksRaycasts = true; }
            else _panelFade = StartCoroutine(FadeCanvasGroup(selectionPanel, 0f, 1f, fadeDuration, setInteractable: true));
        }

        private void HideSelectionPanel(bool instant)
        {
            if (selectionPanel == null) return;
            if (_panelFade != null) StopCoroutine(_panelFade);
            if (instant) { selectionPanel.alpha = 0f; selectionPanel.interactable = false; selectionPanel.blocksRaycasts = false; selectionPanel.gameObject.SetActive(false); }
            else _panelFade = StartCoroutine(FadeCanvasGroup(selectionPanel, 1f, 0f, fadeDuration, setInteractable: false, deactivateOnComplete: true));
        }

        private void ShowConversationHUD(bool instant)
        {
            if (conversationHUD == null) return;
            if (_hudFade != null) StopCoroutine(_hudFade);
            conversationHUD.gameObject.SetActive(true);
            if (instant) { conversationHUD.alpha = 1f; conversationHUD.interactable = true; conversationHUD.blocksRaycasts = true; }
            else _hudFade = StartCoroutine(FadeCanvasGroup(conversationHUD, 0f, 1f, fadeDuration, setInteractable: true));
        }

        private void HideConversationHUD(bool instant)
        {
            if (conversationHUD == null) return;
            if (_hudFade != null) StopCoroutine(_hudFade);
            if (instant) { conversationHUD.alpha = 0f; conversationHUD.interactable = false; conversationHUD.blocksRaycasts = false; conversationHUD.gameObject.SetActive(false); }
            else _hudFade = StartCoroutine(FadeCanvasGroup(conversationHUD, 1f, 0f, fadeDuration, setInteractable: false, deactivateOnComplete: true));
        }

        private static IEnumerator FadeCanvasGroup(CanvasGroup cg, float fromAlpha, float toAlpha, float duration, bool setInteractable, bool deactivateOnComplete = false)
        {
            cg.alpha = fromAlpha; cg.interactable = false; cg.blocksRaycasts = false;
            float elapsed = 0f;
            while (elapsed < duration) { elapsed += Time.deltaTime; cg.alpha = Mathf.Lerp(fromAlpha, toAlpha, Mathf.Clamp01(elapsed / duration)); yield return null; }
            cg.alpha = toAlpha;
            if (setInteractable) { cg.interactable = true; cg.blocksRaycasts = true; }
            if (deactivateOnComplete && toAlpha <= 0f) cg.gameObject.SetActive(false);
        }

        private bool ValidateReferences()
        {
            bool ok = true;
            if (selectionPanel == null)    { Debug.LogError($"{LogPrefix} selectionPanel not assigned.",    this); ok = false; }
            if (rotationUI == null)        { Debug.LogError($"{LogPrefix} rotationUI not assigned.",        this); ok = false; }
            if (characterSwitcher == null) { Debug.LogError($"{LogPrefix} characterSwitcher not assigned.", this); ok = false; }
            if (convaiActivator == null)   { Debug.LogError($"{LogPrefix} convaiActivator not assigned.",   this); ok = false; }
            if (!ok) enabled = false;
            return ok;
        }

        private void DisableConvaiChatPanelBackground()
        {
            GameObject canvas = GameObject.Find("Convai Transcript Canvas - Mobile Chat");
            if (canvas == null) return;
            Transform panel = canvas.transform.Find("Panel");
            if (panel == null) return;
            Image bg = panel.GetComponent<Image>();
            if (bg != null) bg.enabled = false;
        }
    }
}

Please describe the issue in a normal message, and for different issues, create a separate post. It’s difficult to understand the problem as it is now.

my problem is that I have 5 NPC sitting in a carousel when I choose for example NPC 1 NPC 2 answers or NPC 5 I need the right npc to answer - When I go back to the selector screen and choose another the same thing happens

This beta version does not support multiple NPCs.

1 Like