Unable to use Constrained beam search with google/flan-t5-base

🤗Transformers
1

Hi,

I am not able to use constrained beam search with flan-t5 models. I was able to use it with GPT-2 models.

The following code results in error - ‘ValueError: Each list in nested_token_ids can’t be a complete subset of
another list, but is [[3, 7, 11763], [3, 7, 11763, 7], [25471], [3, 7, 11763,
15, 26]].’.

Can you please suggest some solutions to this problem?

`from transformers import AutoTokenizer, AutoModelForCausalLM, AutoModelForSeq2SeqLM

from transformers import GPT2LMHeadModel, GPT2Tokenizer

model = AutoModelForSeq2SeqLM.from_pretrained(“google/flan-t5-base”)
tokenizer = AutoTokenizer.from_pretrained(“google/flan-t5-base”)

force_word = “scared”
force_flexible = [“scream”, “screams”, “screaming”, “screamed”]

force_words_ids = [
tokenizer([force_word], add_special_tokens=False).input_ids,
tokenizer(force_flexible, add_special_tokens=False).input_ids,
]

starting_text = [“The soldiers”, “The child”]

input_ids = tokenizer(starting_text, return_tensors=“pt”).input_ids

outputs = model.generate(
input_ids,
force_words_ids=force_words_ids,
num_beams=10,
num_return_sequences=1,
no_repeat_ngram_size=1,
remove_invalid_values=True,
)

print(“Output:\n” + 100 * ‘-’)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
print(tokenizer.decode(outputs[1], skip_special_tokens=True))

`

2

Hi @elita1995, have you solved your problem?