Hi all, I’ve posted in the Unreal section a few times now. I’ve been looking into reducing latency in response times for my Convai project that currently exists in Unreal Engine, being pushed to the Meta Quest 3. Current latency is around the 3-4 second mark before a response is returned which according to this comment back in June is the best latency times we can realistically expect, but that an update was in the works to significantly reduce latency times. Seeing as quite a few months have passed since this comment, I wanted to ask if this update has been made available for Unity, and, if so, if this update supports Android.
As for why Unity when I’ve been working with Unreal, I haven’t been able to find a fix for the passthrough issue I mentioned in this post and have had to use VR instead of the preferred AR. According to Convai documentation here, Unity has more explicit support for MR, so I’m investigating moving my project to Unity to bypass the passthrough issue, and if reduced latency can be expected as well, that would be wonderful.
Great news. I am running into issue with installation, though: The documentation says that to install through UPM, I should select install package by name and paste incom.convai.convai-sdk-for-unit.When I do so, however, I get the following error:
[Package Manager Window] Error adding/removing packages: com.convai.convai-sdk-for-unity.
Unable to add packages:
Unable to add package [com.convai.convai-sdk-for-unity]:
Cannot download the package. Your account does not grant permission to use the package. Please visit the Asset Store to acquire the package and add it to your account:
Access to ‘https://download.packages.unity.com/com.convai.convai-sdk-for-unity/-/com.convai.convai-sdk-for-unity-0.1.0.tgz’ was denied
UnityEditor.EditorApplication:Internal_CallUpdateFunctions ()
Navigating to the asset store shows me what I assume to the non-beta version of the plugin. I have this version installed and properly set up, but there are discrepancies between the documentation and what I’m seeing in Unreal, hence the assumption. Notably, I’m failing to find any documentation for transitioning to hands free conversation.
That worked for me, thank you. I’ve gotten my project set up with the latest 4.0 beta plugin, and I’m noticing that latency actually got 2-3x worse, taking 7-9 seconds for a response up from 2.5-3.5s when compared to the stable version of the plugin. The lipsync component is set to the balanced latency mode, and I believe my Convai character it set up to use the lowest latency options available: gemini-2.5-flash-lite for the response model and an ElevenLabs voice, which consistently returned responses faster than the recommended GCP voices. My character ID is dd30589c-1342-11f1-843d-42010a7be02c. Is this known behavior, or am I possibly doing something wrong?
Thanks for the update. I’ve tested with gpt-realtime-1.5 and gpt-realtime-mini, and both are returning responses in the 2.5-3 second range, with mini being maybe 10ms faster, so much much better.
It’s the same ID as before, dd30589c-1342-11f1-843d-42010a7be02c. I’ve added text to the speaking style section as opposed to just the core description since the last time I was was doing latency testing, so that may be reflecting a slight latency increase.