How to Retrieve Live Website Data

Integrations Unity SDK
1

What do you need help with?

Question

Installation method

UPM

Issue area

Other

Impact

Blocker - I cannot continue

Unity version

6000.4.10f1

Convai Unity SDK version

Convai 4.6

Target platform

Windows

Where does it happen?

Not tested yet

Short summary

How to Retrieve Live Website Data

What happened?

Hi everyone,

I’m looking into possible solutions for allowing my character to retrieve information from a website without having to constantly update the Knowledge Bank.

I’ve seen that this can potentially be done using an External API. However, the specific website I’m interested in does not allow third-party applications to directly retrieve its information.

Is there any alternative approach that could work in this case? For example, would it be possible to have the character crawl or otherwise retrieve the data from the website through another method?

I’d really appreciate any suggestions or guidance. A diagram or example of the possible architecture/workflow would also be extremely helpful.

Thanks in advance!

I checked the relevant documentation, resources, and similar forum posts before posting.

on

I confirm that I did not include API keys, tokens, passwords, secrets, private character data, private conversation logs, or customer data.

on

2

Could you share a bit more information about the website you’re trying to use?

What kind of website is it, and what type of data are you trying to retrieve from it? Also, when you say it doesn’t allow third-party applications to access the information, do you mean it doesn’t provide an API, or does it actively block automated access?