Using model() instead of model.generate()

[Vidushee]

Can you explain a bit more about your answer ? isn’t it just calling the forward pass ?
And how about this ?

Generate without using the generate method

Posting this here for visibility. What if you want to decode the output of a generative seq2seq model (like T5, BART, etc.) yourself, without using the .generate() method? The code example below illustrates this. Suppose that the model is given a long text, for which it needs to generate a summary. We illustrate here how to manually decode the generated ids autoregressively. In each iteration, we add the predicted token id by the model to the decoder_input_ids, which are then fed as input to th…