Hello everyone,
We are back with a comprehensive new tutorial for our Unity developers! If you are looking to build fully interactive AI characters with lifelike facial animation and incredibly low latency, this step-by-step guide is for you.
In this video, we walk you through how to set up an interactive AI character in Unity using Convai’s updated WebRTC-based Unity plugin.
What Makes This Update Special?
Featuring our demo character Susan, this tutorial showcases how you can achieve natural, fluid conversations inside Unity. The updated plugin enables:
-
WebRTC Voice Interaction: Experience significantly lower response latency for fluid turn-taking and realistic exchanges. -
Hands-Free Conversations: Utilize Voice Activity Detection (VAD) for natural dialogue without push-to-talk inputs. -
NeuroSync Integration: High-quality, real-time facial animation and accurate lip sync powered by Convai’s in-house AI model. -
Deep Context: Build smarter characters with long-term memory and multimodal knowledge integration.
What You Will Learn
We cover the entire pipeline from scratch so you can easily integrate lifelike AI avatars into your games, training simulations, and XR experiences:
-
Installation: How to install the Convai SDK via the Unity Package Manager (UPM) or the Unity Asset Store.
-
Core Setup: Configuring API keys, the Convai Manager, Player, and Character components.
-
UI Integration: Enabling a transcript UI for in-scene conversations.
-
Facial Animation: Setting up real-time lip sync with NeuroSync on a custom avatar.
Resources & Downloads
Ready to build your own intelligent, lip-syncing avatars? Grab the plugin and check out the documentation below:
-
Get the Plugin: Convai Unity Plugin (Unity Asset Store)
-
Setup Documentation: Latest Unity Plugin Beta Overview
-
Installation Guides: Install via UPM | Install via Asset Store
-
Deep-Dive Blog Post: Bringing Interactive AI Avatars to Unity Projects
If you run into any issues during setup or want to share your completed projects, let us know in the replies below.
Happy developing!