I’m using model.generate() for text generation. I’m wondering is there a way to whitelist specific tokens so that they are returned during the beam search phase. For example - I want to the “force” the response to contain a question mark or a speccific phrase

Whitelist specific tokens in beam search

100worte March 16, 2021, 7:13pm 4

You can pass a LogitsProcessorList to beam_search. For some concrete examples of LogitProcessors, refer to generation_logits_process.py.

1 Like

Use custom LogitsProcessor in `model.generate()`

Topic		Replies	Views
Use custom LogitsProcessor in `model.generate()` Beginners	2	4426	March 14, 2023
Prevent repeat tokens in GPT2LMHeadModel text generation with max_new_tokens=1 Beginners	0	911	November 19, 2021
Logits from generate and model call different 🤗Transformers	0	42	April 14, 2024
Showing individual token and corresponding score during beam search Beginners	5	2953	November 28, 2023
Custom Decoding Strategy Beginners	0	187	December 6, 2023