From Dialogue to Action: Create Scene-Aware AI Characters That Act AND React | Convai Unreal Engine

Community Guides & Tutorials
, ,
1

Hello everyone,

We are excited to share an updated guide in our Convai Actions series for Unreal Engine 5! In this tutorial, we show you how to enable Convai Actions specifically for MetaHuman avatars, taking them beyond basic conversation and allowing them to translate player intent into actual gameplay behavior in real time.

With built-in actions, your MetaHuman can move, follow, stop, point, gesture, and interact with physical objects in your scene using simple, natural language commands.

:sparkles: What You Will Learn in This Video

Starting with a MetaHuman avatar that already has Convai integrated, we take you through the entire setup required to give your character spatial awareness and agency:

  • :person_running: Built-In Actions: Enable actions like Move To, Follow, Stop, Point, and Gesture directly from the Convai Chatbot component.

  • :world_map: Navigation Setup: Configure the Nav Mesh Bounds Volume and set up Convai Pawn Movement so your MetaHuman navigates your level smoothly.

  • :package: Scene & Object Awareness: Learn how to add references to objects in your environment (using Convai Object components) so your AI understands commands involving specific scene items—like pointing to a weapon or moving to a target object.

  • :brick: Foundation for Advanced Workflows: Set up the groundwork for upcoming tutorials covering parameterized actions, Dynamic Context, and custom Blueprint-driven behaviors.

:link: Resources & Prerequisites

Whether you are starting from scratch or continuing your MetaHuman setup, check out the resources below:

Prerequisite & Setup Tutorials:

Documentation & Blog:

We are excited to see how you bring your MetaHumans to life with scene-aware actions! Drop your questions, feedback, or project show-and-tells in the replies below.

Happy developing!