Original Discord Post by chescored | 2024-08-24 08:40:54
I have managed to implement Convai with a custom NPC in Unity and the Meta SDK (NOT OpenXR). I have also added lip synchronization through AudioSource and GetOutputData so that it moves the lips depending on the amplitude of the audioClip. I also have a customized multiplier so that he opens his mouth more. In the example, it is not very high.
Reply by boomslanghegemony | 2024-08-26 18:10:43
Definitely look into a Unity asset called ‘Salsa’ by Crazy Minnow Studios. I use it all the time. An excellent way to make your avatars speak and emote. All you do is put the audio clip into an Audio Source. Helps if your characters are Daz, Ready Player Me or CC4.
Replying to boomslanghegemony’s Message
Reply by boomslanghegemony | 2024-08-26 18:10:43
Definitely look into a Unity asset called ‘Salsa’ by Crazy Minnow Studios. I use it all the time. An excellent way to make your avatars speak and emote. All you do is put the audio clip into an Audio Source. Helps if your characters are Daz, Ready Player Me or CC4.
Reply by chescored | 2024-08-27 14:11:46
Oh, thank you, in my case since these are anime cartoon characters I don’t need perfect mouthing. In the video I have not yet adjusted the mouth opening properly and I have not added the blinking either. I got the code that I used to use the audiosource with a Gemini Google AI that was quite simple and enough for me, if you want me to pass it on to you, let me know, greetings.