Play videos on command

Unity Questions
,
1

Is there a way to make the NPC “show” you a video on a media player on user request?

E.G:

“Can you show me a video of the last show?”

“Sure”

Media player pops up with the video

Keep in mind that the NPC will stand still and never moves.

2

Hello @Luca_Trianti,

Welcome to the Convai Developer Forum!

You can use our Action Feature.

3

Thanks! But is there a way to know what video to play based on user request?

4

This depends on how you’ve implemented your PlayVideo action. You’ll need to determine which video to play based on your implementation.

Alternatively, you can use our Narrative Design feature. You can configure sections corresponding to each video, and based on the user input, the relevant section will be triggered,where you can initiate the video accordingly.

5

Thanks! I actually solved this issue with the narrative design and some scripting (by detecting with API calls when the NPC replied in specific ways). Thanks for your help!

6

@Luca_Trianti Awesome!

7

@Luca_Trianti Do you want to share your solution so that other users can use it too?

8

@Luca_Trianti Is it possible for you to share your solution with the community on how you managed to do the action for video player?

9

So, there are two main classes: one being the “media handler” and the other one being the “api handler”.

Info:

  • One script handles the API: sends user text/audio to Convai, receives the AI’s response, plays the voice audio if included.
  • The other one intercepts the AI’s response text, looks for patterns like “video of…” or other language variants, and uses fuzzy matching to find the closest local video listed in avideoList.txt file (containing the video file names, and I know, it’s not very dynamic).
  • If a match is found, it shows the video UI and plays the corresponding video from the local folder.
  • Video titles are loaded asynchronously from videoList.txt in StreamingAssets/Videos, making it easy to update the list without recompiling.
  • For fuzzy matching, I used FuzzySharp (NuGet package) to compare the requested video name with available titles and trigger playback when the match confidence exceeds a threshold (I set it at 30%).
  • I handle platform differences by copying the video to Application.persistentDataPath before playing it, which is especially needed on Android where direct access to StreamingAssets isn’t possible.
  • Commands work across multiple languages by checking for keywords like:
  • “video of” (English)
  • “video di” (Italian)
  • “vidéo de” (French)
  • etc.
  • Example workflow:
  1. User says: “I’d like to see the video of -insert video title here-.”
  2. Convai responds: “Here is the video of -insert video title here-.”
  3. The media handler parses “video of -insert video title here-”, matches it to the closest video, and plays it automatically.

Here are the links to the 2 classes:

The ConvaiAPIHandler is attached to an empty game object, while the ConvaiMediaHandler is attached directly to the canvas of the video player.

If you have any questions please ask.