I’m experiencing response times from the avatar of about 10 seconds (counting out loud), please see the video : https://youtu.be/s-nO9zf3v3U
In the Playground the same character (37d2b31c-7cea-11ef-bfad-42010a7be011) responds in 6 to 7 seconds.
I’d like to decrease this but I’m not sure how? My internet is reliable and reasonably fast. What things affect response times? Does the amount of knowledge base info? Size of backstory? I think EL voices increase it, do OpenAI voices increase it also (I use these)? Do the amount of Narrative Design triggers/sections increase it?
I’m not complaining, I really like Convai and their service, I’m just trying to improve aspects of my app.
Yeh I don’t think my app is at fault, because the latency in the Playground is still 6 or 7 seconds, I asked a friend who also has a Convai app to test 1 of his characters’ latency in the Playground and it was 1 to 3 seconds, that’s what the latency used to be for me.
Some users are starting to get on my case about it, I’ve told them I’ve informed Convai. If there’s anything I can do while you guys are investigating it please let me know. Thank you.
Hey @Tyke, could you please have two conversations from both playground and your app, with the same messages and character? This will help us investigate the inconsistency. A small conversation, lets say around 5 identical messages would be great!
Hi @ayush I’ve done as you’ve requested. Please see :
(conversation in my app)
and
(conversation in the playground)
In my app when I ask what is the day, it fails because of something in my app, so please ignore that bit.
In these video’s an OpenAI voice is used, I tried a non OpenAI voice and the latency seemed to be reduced, not sure if that is relevant.
If you need anymore info from me please let me know. The character ID used here is 37d2b31c-7cea-11ef-bfad-42010a7be011
EDIT: just for reference, this was the reply time in my app a few months ago, to show it used to be much lower (I recall the Playground reply time being lower too back then) :
I would really appreciate being able to lower the latency, if it’s not possible then I will live with it but it does negatively affect the flow of conversation between user and avatar.
I acknowledge my app will add some latency but as you can see in the video, even in the Playground the latency is high. EDIT: you can also see in the video my internet connection is reasonably fast during these tests.
For optimising latency
Voices: GCP/Azure
LLMs: Fastest would be the smaller models(gemini flash, 4o-mini), but they might not have the full feature set. 4o is next and has more features. But these might not support your use-case.
Size of the backstory and narrative design sections does affect the response time as well
Thank you for the reply. I will look into streamlining things such as backstory, narrative design, I assume knowledge base size affects things too, I can try to condense that. Will also try other LLM’s.
Yeh can confirm that how much stuff the character has as in backstory, narrative design, knowledge, etc, really makes a difference. I’ve just created a new test character and left everything almost blank and response times are about halved. Time to do some streamlining!
Just in case anyone’s interested - my response times were increased because my app had become a bit bloated i.e. I wasn’t using Narrative Design efficiently and my backstories and knowledge base’s had alot of fluff in them that wasn’t needed.
I’ve streamlined them (still in the process of doing so) and so far have reduced response times by about 3 to 4 seconds (was about 9, it’s now about 5), still more to do.
