UE5.6 — Beginner help: my own Player + several NPCs (talk/listen/walk/idle, lipsync, “look to talk”) — VR training project

Context

I’m building a VR training prototype in Unreal Engine 5.6 (Windows + Quest). I’m using my own custom Player (not ConvaiBasePlayer). My Player has a Convai Player component. Each NPC is a Character with Convai Chatbot. NPCs are Reallusion CC5 meshes exported with facial shapes; in Unreal I enabled Import Morph Targets. I’d love a very beginner-friendly checklist to get the basics working.

I can’t get my NPCs to animate. They canorlk, but there’s no lip sync or gestures as they talk — just idle animation and the voice!

What I need (simple steps or screenshots, please):

  1. Player (custom pawn)

    • What do I click/add so I can press a key (or VR button) to start/stop talking and have the conversation actually happen?
  2. NPCs switch animations

    • Right now they only idle. How do I make them talk, listen/think, and walk at the right times, using the simplest possible steps?
  3. Lipsync

    • With CC5 facial shapes exported and morph targets imported, what exact steps make the mouth move when the NPC talks?
  4. Multiple NPCs in one scene

    • Do I need to set anything so each NPC knows who the Player is?

    • Can all NPCs reuse the same animation setup, or do I need separate ones per NPC?

  5. “Look at to talk” (gaze targeting)

    • I want to look at an NPC and press a key to talk to that one (not all). What’s the simplest way to set that up?

If you have a tiny sample or beginner checklist for: custom Player + two NPCs + gaze targeting, that would help me finish my mid-October prototype. Thanks so much!

Hello,

Welcome to the Convai Developer Forum!

Many of your questions are covered in our docs and forum. For a minimal setup without changing your parent classes, follow this guide:
https://docs.convai.com/api-docs/plugins-and-integrations/unreal-engine/guides-v2-under-development/getting-started/simple-talking-cube

CC5 is not officially supported at the moment. For animations, use the Convai Animation Blueprint. Since CC5 is unofficial, you may encounter issues. For more details, see our Reallusion video:
https://youtu.be/UyxNliF8LKU?si=EaeM17xd-xVK8eJw

To help NPCs recognize context or each other, use Dynamic Environment Info, Actions, or refine character Descriptions:
https://docs.convai.com/api-docs/plugins-and-integrations/unreal-engine/guides/dynamic-environment-info