This is more like a feature question

[K3]

Original Discord Post by giorob._38688 | 2024-10-10 16:14:13

Is lip sync achievable for Ready Player Me characters in the browser?

Would a tutorial like this one below explain the right way to achieve it?

Lip Sync - React Three Fiber Tutorial

Embedded Content:
Lip Sync - React Three Fiber Tutorial
Let’s learn how to add LipSync on Ready Player Me with React Three Fiber and Three.js

We’ll discover the concept of MorphTargets and Visemes

Live demo
https://r3f-lipsync-tutorial.vercel.app/

Get the starter pack here

GitHub - wass08/r3f-vite-starter: A boilerplate to build R3F projects

Final Code

GitHub - wass08/r3f-lipsync-tutorial

threejs #r3f #lips…
Link: https://www.youtube.com/watch?v=egQFAeu6Ihw

[K3]

Reply by k3kalinix | 2024-10-10 16:22:14

Hello <@1216669143001202768> ,

[K3]

Reply by giorob._38688 | 2024-10-10 16:23:13

hello! <@1023671043287699568>

[K3]

Reply by saurav0037 | 2024-10-10 16:23:24

<@1216669143001202768> You can try this tutorial for learning about how morph targets work

[K3]

Reply by giorob._38688 | 2024-10-10 16:25:13

which tutorial?and would this help me with getting my characer to lipsync what she says in our browser demo?

[K3]

Reply by saurav0037 | 2024-10-10 16:26:54

Oh i though you are asking about the above tutorial my bad

[K3]

Reply by saurav0037 | 2024-10-10 16:29:17

There is no tutorial for lipsync specifically but you can go through this : https://www.youtube.com/watch?v=5tYyeXPo5vg&t=491s just for reference on how things work. Plus the code you are looking for is here:

1. RPM : GitHub - Conv-AI/RPM-Lipsync
2. Actor core: GitHub - Conv-AI/Reallusion-web

Embedded Content:
AI Characters in Web Browser using ThreeJS and Reallusion Character…
Convai powers the minds of 3D avatars in virtual worlds. In this video, we’ll be guiding you on integrating Convai’s AI functionalities with a @reallusion Avatar, all on your web browser using Three.js and React. Using Convai, you can effortlessly add complex AI functionalities to your Reallusion character, enabling dynamic natural conversations…
Link: https://www.youtube.com/watch?v=5tYyeXPo5vg&start=491

GitHub - Conv-AI/RPM-Lipsync
Contribute to Conv-AI/RPM-Lipsync development by creating an account on GitHub.
Link: GitHub - Conv-AI/RPM-Lipsync

GitHub - Conv-AI/Reallusion-web
Contribute to Conv-AI/Reallusion-web development by creating an account on GitHub.
Link: GitHub - Conv-AI/Reallusion-web

[K3]

Reply by giorob._38688 | 2024-10-11 06:32:19

Ok thanks guys but it seems like from the videos you’ve sent me that in order to have lipsyncing I’ll need to use a reallusion character, rather than a Ready Player Me one, can you confirm this?

[K3]

Reply by saurav0037 | 2024-10-11 17:08:07

<@1216669143001202768> I shared a repo with RPM-Lipsync

[K3]

Reply by saurav0037 | 2024-10-11 17:08:16

you can use that for ready player me