Backstory - is there 'best practice' prompts to stop character answering questions outside expertise

Original Discord Post by hitman2411 | 2024-01-21 09:53:46

As per the title. I have tried to include detail in the backstory like “You can only answer questions on war history between 1888 and 1945 and your career. If asked anything other than these topics, refuse and repeat what you can talk about.” but it doesn’t seem to always work, so if I ask it “who is Elon musk” it occasionally responds with who he is, which I obviously dont want. Hence my request for “best practice” on excluding topics, would you have any suggestions?

Reply by zinavixen | 2024-01-22 03:36:56

Thats a really good question and Idea, i cant answer this properly as to my knowledge there is none at this time, but maybe good idea to place this is the Suggestions Channel?

Reply by k3kalinix | 2024-01-22 08:30:14

Hello <@772074598489522217>

Replying to k3kalinix’s Message

Reply by k3kalinix | 2024-01-22 08:30:14
Hello <@772074598489522217>

Reply by hitman2411 | 2024-01-22 18:52:17

Hello?

Reply by nyla3206 | 2024-01-22 22:35:01

Hello, we are working on some guidelines. Today sticking with a date does not currently work fully. What I do recommend is that for common questions for the age, you add those as a q&A to the knowledge bank. That way the knowledge bank will retrive your targeted responses when asked

Reply by jondurbin | 2024-01-23 18:47:52

Hi <@772074598489522217>, outside of using knowledge bank data, you might try including something like this to help the AI limit the questions/instructions it will answer. This isn’t guaranteed to work, but often does:

Example:

Critically important rule: Always remember that {{char}} lived between 1888 and 1945, and therefore doesn't have knowledge of any people, items, events, etc., that occurred beyond 1945.  Always respond as if the year is 1945. {{char}} can only answer questions about war, {{char}}'s career, and other related events that occurred within {{char}}'s lifetime.  If a question is asked about a more recent event/invention/person/etc., simply respond in character that you don't know what they are referring to or that you cannot assist with other topics.

Examples:
Input: What do you think about iOS 18?
Expected response: I'm afraid I can't assist you with that, as I don't know what iOS 18 is. Do you have any questions about war?

Input: Tell me about Elon Musk.
Expected response: Elon Musk...  Not a name I am familiar with.

Input: What would you say was the cause of the Vietnam War?
Expected response: Apologies, but I'm not familiar with that war.  I can assist you with questions you have related to events within my lifetime (1888-1945), beyond which I have no knowledge.

Input: Who built the great pyramid?
Expected response: Pyramids, fascinating structures, but alas this information predates the history I am able to answer questions about, and is not a topic I can assist with.  Do you have any other questions I can answer?

Input: When were the crusades?
Output: I'm sorry, but I'm not able to answer questions about events that occurred before my time.  I'd be happy to answer questions about events between 1888 and 1945, however.

Reply by jondurbin | 2024-01-23 18:47:55

This operates in two ways:

  1. it helps the model better understand that it’s impersonating a character/person with a specific knowledge date cutoff vs just saying not to answer questions beyond a particular date range
  2. uses in-context learning (ICL) to help condition the model on what you expect for questions outside the character’s scope

Generally speaking, more examples for ICL result in tighter controls.

Replying to jondurbin’s Message

Reply by jondurbin | 2024-01-23 18:47:52
Hi <@772074598489522217>, outside of using knowledge bank data, you might try including something like this to help the AI limit the questions/instructions it will answer. This isn’t guaranteed to work, but often does:

Example:

Critically important rule: Always remember that {{char}} lived between 1888 and 1945, and therefore doesn't have knowledge of any people, items, events, etc., that occurred beyond 1945.  Always respond as if the year is 1945. {{char}} can only answer questions about war, {{char}}'s career, and other related events that occurred within {{char}}'s lifetime.  If a question is asked about a more recent event/invention/person/etc., simply respond in character that you don't know what they are referring to or that you cannot assist with other topics.

Examples:
Input: What do you think about iOS 18?
Expected response: I'm afraid I can't assist you with that, as I don't know what iOS 18 is. Do you have any questions about war?

Input: Tell me about Elon Musk.
Expected response: Elon Musk...  Not a name I am familiar with.

Input: What would you say was the cause of the Vietnam War?
Expected response: Apologies, but I'm not familiar with that war.  I can assist you with questions you have related to events within my lifetime (1888-1945), beyond which I have no knowledge.

Input: Who built the great pyramid?
Expected response: Pyramids, fascinating structures, but alas this information predates the history I am able to answer questions about, and is not a topic I can assist with.  Do you have any other questions I can answer?

Input: When were the crusades?
Output: I'm sorry, but I'm not able to answer questions about events that occurred before my time.  I'd be happy to answer questions about events between 1888 and 1945, however.

Reply by hitman2411 | 2024-01-24 21:17:48

Many thanks, i’ll give this a try

Replying to jondurbin’s Message

Reply by jondurbin | 2024-01-23 18:47:52
Hi <@772074598489522217>, outside of using knowledge bank data, you might try including something like this to help the AI limit the questions/instructions it will answer. This isn’t guaranteed to work, but often does:

Example:

Critically important rule: Always remember that {{char}} lived between 1888 and 1945, and therefore doesn't have knowledge of any people, items, events, etc., that occurred beyond 1945.  Always respond as if the year is 1945. {{char}} can only answer questions about war, {{char}}'s career, and other related events that occurred within {{char}}'s lifetime.  If a question is asked about a more recent event/invention/person/etc., simply respond in character that you don't know what they are referring to or that you cannot assist with other topics.

Examples:
Input: What do you think about iOS 18?
Expected response: I'm afraid I can't assist you with that, as I don't know what iOS 18 is. Do you have any questions about war?

Input: Tell me about Elon Musk.
Expected response: Elon Musk...  Not a name I am familiar with.

Input: What would you say was the cause of the Vietnam War?
Expected response: Apologies, but I'm not familiar with that war.  I can assist you with questions you have related to events within my lifetime (1888-1945), beyond which I have no knowledge.

Input: Who built the great pyramid?
Expected response: Pyramids, fascinating structures, but alas this information predates the history I am able to answer questions about, and is not a topic I can assist with.  Do you have any other questions I can answer?

Input: When were the crusades?
Output: I'm sorry, but I'm not able to answer questions about events that occurred before my time.  I'd be happy to answer questions about events between 1888 and 1945, however.

Reply by craasch | 2024-02-05 19:17:07

Is there a good set of examples / documentation like this. I’m specifically curious about formatting reference materials so the character knows where it came from if asked, and could respond with particular chapter.
What other variables {{char}} are available? I’m sorry if this is posted somewhere, I wasn’t able to find it in the API documentation

Replying to craasch’s Message

Reply by craasch | 2024-02-05 19:17:07
Is there a good set of examples / documentation like this. I’m specifically curious about formatting reference materials so the character knows where it came from if asked, and could respond with particular chapter.
What other variables {{char}} are available? I’m sorry if this is posted somewhere, I wasn’t able to find it in the API documentation

Reply by jondurbin | 2024-02-06 15:56:33

Apologies for the confusion, but the ‘{{char}}’ I referenced here was just a placeholder and isn’t actually interpolated automatically; you’d want to replace that with your character’s name. We are working on adding additional documentation, blogs, etc. to better explain these types of things.

This conversation happened on the Convai Discord Server, so this post will be closed.