Hi! It seems like some models implement both functions and semantically they behave similarly, but might be implemented differently? What is the difference? In both cases, for an input sequence, the model produces a prediction (inference)? Thank you, wilornel

Hi, forward() can be used both for training and inference. Forward refers to a single forward pass through the network. During training, we apply a forward pass to get the model’s predictions, and then do a backward pass to compute the gradients of the parameters with respect to the loss, which we…

What is the difference between forward() and generate()?

Beginners

nielsr December 25, 2023, 9:08pm 4

The generate method is more feature complete with various fancier decoding methods besides greedy decoding, such as beam search and top-k sampling.

Topic		Replies	Views
T5 forward pass versus generate, latter outputs non-sense Beginners	8	2914	March 25, 2021
The output of T5 is not consistent on multiple sequences 🤗Transformers	1	870	May 11, 2022
Is model.generate slower than model forward call? 🤗Transformers	1	191	August 18, 2024
Seq2SeqTrainingAguments Beginners	0	263	January 26, 2023
Disparity between output from `forward` and `generate` for greedy search (using Whisper) 🤗Transformers	3	1358	August 11, 2024

What is the difference between forward() and generate()?

Related topics