I’m currently using HuggingFace’s T5 implementation for text generation purposes. More specifically, I’m using the T5ForConditionalGeneration to solve a text classification problem as generation. The model’s performance is overall very satisfactory after training, but what I am wondering is how I c…

You can use the arguments output_scores=True, return_dict_in_generate=True in generate

How to get the logits for the T5 model when using the `generate` method for inference?

Vignesh1997 April 18, 2023, 5:09am 4

you can use compute_transition_scores to get scores of each token
find more info here Generation

Topic		Replies	Views
-inf values for logit score outputs with model.generate 🤗Transformers	3	853	January 2, 2025
Can I get logits for each sequence I acqired from model.generate()? Beginners	1	1315	November 27, 2020
How can I obtain the logits via model.generate()? 🤗Transformers	2	3067	October 8, 2024
T5ForConditionalGeneration, How to get prediction probabilities or logits at the inference time? (to calculate perplexity) 🤗Transformers	0	694	April 5, 2022
TFT5ForConditionalGeneration generate returns empty output_scores 🤗Transformers	1	402	October 28, 2024