Posting this here for visibility. What if you want to decode the output of a generative seq2seq model (like T5, BART, etc.) yourself, without using the .generate() method? The code example below illustrates this. Suppose that the model is given a long text, for which it needs to generate a summary.…

Generate without using the generate method

espoir September 15, 2023, 8:02am 7

This may help a lot. What if the decoding is using beam search?

Topic		Replies	Views
Using model() instead of model.generate() 🤗Transformers	3	525	January 30, 2025
Encoder-Decoder model only generates bos_token's [<s><s><s>] Models	17	3176	December 6, 2022
Using generate() method with decoder Models	0	569	January 16, 2022
Rewriting generate function for manual decoder input 🤗Transformers	7	3567	July 11, 2022
Control EncoderDecoderModel to generate tokens step by step 🤗Transformers	8	2607	June 8, 2022