Overview
I am new to the transformers-library and I want to generate text using GPT2 while influencing its output. This I do via the PhrasalConstraint that lets me add certain tokens that are to apear in the sentence generated by GPT2.
Mainly I oriented myself on this blogpost.
Problem
Now the problem is, that the constraint words will almost certainly appear after the input or at the end of the generated sentence. Never in the middle of the sentence.
→ I suspect that the constraints do not fit the generated sentence, and are therfore “discarded” to the end/ beginning, to keep up the overall probability of the generation.
Example
input: I like to
constraint: France
output_1 → I like to France is a beautiful county
output_2 → I like to become a carpenter when I grow up France
Does anybody know how I can improve this? Is there a way to make the constrains fit more natural into the generated sentence? Or is there another way to infuse constraints into the generation, that you are aware of?
Thanks for any help!
Code
from transformers import AutoModelForCausalLM, AutoTokenizer, PhrasalConstraint
tokenizer = AutoTokenizer.from_pretrained("dbmdz/german-gpt2",add_prefix_pace=True)
# the model is a german version of gpt2
model = AutoModelForCausalLM.from_pretrained("dbmdz/german-gpt2")
tokenizer.add_special_tokens({"pad_token": "[PAD]"})
encoder_input_str = " Im Krankenhaus musste ihr ein"
force_words = [" Morphium"]
constraints = []
for constraint_entitiy in force_words:
constraints.append(PhrasalConstraint(tokenizer(constraint_entitiy).input_ids))
input_ids = tokenizer(encoder_input_str, return_tensors="pt").input_ids
force_words_ids = tokenizer(force_words, add_special_tokens=False).input_ids
outputs = model.generate(
input_ids,
constraints = constraints,
num_beams=10,
num_return_sequences=1,
no_repeat_ngram_size=1,
min_length=20,
)
print(tokenizer.decode(outputs[0]))