Finetuning of conversational model without train data in conversation style

Intermediate
1

Hi all,

I want to finetune a Language Model to obtain a chatbot that imitates a certain person in a conversation. The problem is, I only have quotes of that person as train data, no conversation format. So I guess finetuning a generative model that doesnt know how to answer to question resp. participate in a conversation is no option. What about finetuning a conversational model with train samples in a style like this: you are a chatbot that replies in the style of Donald Duck - “some quote”.
I have no idea if it can work and the model will combine its conversational understanding with its donald duck style knowledge, if I enter a respective prompt.

Any ideas, experiences or recommendations?

Thank you!

2

I think it depends on how much data you have.
It also depends on which model you use. In simple terms, causal language modeling models try to complete documents, so with a certain prompt structure you could trick the model into a kind of conversational document, which the model then wants to complete.
A model that is already fine-tuned towards chat will probably manage this task better.
Maybe prompt engineering with one shot prompts or more shot prompts could help.

You could create a chat template and apply it to the tokenizer.

messages = [
    {
        "role": "system",
        "content": "You are a friendly chatbot who always responds in the style of a donald duck",
    },
    {
		"role": "user",
		"content": "Some Question"
	},
	{
		"role": "assistant",
		"content": "Quack Quack..." # Donald Duck style answer
	},
	{
		"role": "user",
		f"content": "{question}"
	},
 ]

This generates prompts like this:

<|system|>
You are a friendly chatbot who always responds in the style of a donald duck</s> 
<|user|>
Some Question</s> 
<|assistant|>
Quack Quack...</s> # Donald Duck style answer
<|user|>
Your dynamically inserted question </s> 
<|assistant|>
Generated answer..

prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")

For further information: chat templates

I think this could work if the model is supposed to imitate a well-known person from whom there was actually data in the training set.
Alternatively, like you mentioned, you could try to create a data set with this prompt template and fine tune it in that direction.

Unfortunately no guarantee that it will actually work.

1 Like