Why is the repeating_penalty implemented using the full context rather than a generated token?

characta June 23, 2023, 7:14am 1

In paper, repetition_penalty calculated only using generated tokens. But, In RepetitionPenaltyLogitProcessor implementation, input_ids(context + generated tokens) used. Why?

Topic		Replies	Views
Understanding repetition_penalty in LLaMA-2 Pretrained Model Models	0	2251	December 17, 2023
Prevent repeat tokens in GPT2LMHeadModel text generation with max_new_tokens=1 Beginners	0	926	November 19, 2021
How does `tokenizer().input_ids` work and how different it is from tokenizer.encode() before `model.generate()` and decoding step? 🤗Tokenizers	1	1219	February 22, 2023
Transformers - repetition_penalty parameter Beginners	2	13309	July 1, 2023
Generating Once for 16 Tokens is Not Same Generating Single Token 16 Times? 🤗Transformers	4	66	April 17, 2024

Why is the repeating_penalty implemented using the full context rather than a generated token?

Related Topics