Does run_generation.py has a "no_repeat_ngram_size" attribute?

kintaro · September 1, 2020, 10:26pm

Good day,

I’m generating text with run_generation.py. I would like to know if there is any way to pass the attribute no_repeat_ngram_size. I have seen that attribute with model. generate but not in run_generation.py.

valhalla · September 2, 2020, 8:29am

hi @kintaro,
run_generation.py is used for auto-regressive generation which uses sampling.
no_repeat_ngram_size is used for beam search so run_generation.py doesn’t have that attribute since it only uses sampling for generation

yjernite · September 2, 2020, 2:17pm

Hi @kintaro and @valhalla

Thanks @valhalla for replying right away! In this case, no_repeat_ngram_size can actually be applied both in the beam search and sampling generation. I think we just left the option out of the run_generation.py script because we didn’t want it to be too crowded, but you can just pass the argument in:

github.com

huggingface/transformers/blob/master/examples/text-generation/run_generation.py#L252


    )
else:
    encoded_prompt = tokenizer.encode(prompt_text, add_special_tokens=False, return_tensors="pt")
encoded_prompt = encoded_prompt.to(args.device)

if encoded_prompt.size()[-1] == 0:
    input_ids = None
else:
    input_ids = encoded_prompt

output_sequences = model.generate(
    input_ids=input_ids,
    max_length=args.length + len(encoded_prompt[0]),
    temperature=args.temperature,
    top_k=args.k,
    top_p=args.p,
    repetition_penalty=args.repetition_penalty,
    do_sample=True,
    num_return_sequences=args.num_return_sequences,
)

(just add a e.g. no_repeat_ngram_size=3, line in the arguments for the generate function)

valhalla · September 3, 2020, 4:13am

Thanks @yjernite! Didn’t know that no_repeat_ngram_size can also be used with sampling.

kintaro · September 4, 2020, 2:49pm

Thanks a lot for your great help

Topic		Replies	Views
Confused by description: encoder_no_repeat_ngram_size 🤗Transformers	0	679	November 1, 2021
Prevent repeat tokens in GPT2LMHeadModel text generation with max_new_tokens=1 Beginners	0	926	November 19, 2021
Where to set the batch size for text generation? Beginners	0	920	September 17, 2022
Batch_decode does not give the correct output as generate 🤗Transformers	0	247	March 17, 2022
TrOCR repeated generation Beginners	3	990	November 30, 2021

Does run_generation.py has a "no_repeat_ngram_size" attribute?

Related Topics