How to get the logits for the T5 model when using the `generate` method for inference?

I’m currently using HuggingFace’s T5 implementation for text generation purposes. More specifically, I’m using the T5ForConditionalGeneration to solve a text classification problem as generation.

The model’s performance is overall very satisfactory after training, but what I am wondering is how I can get the logits for generation?

I’m currently performing inference as is suggested in the documentation via model.generate(**tokenizer_outputs), but this simply outputs the IDs themselves without anything else.

The reason why I want the logits is because I want to measure the model’s confidence of generation. I’m not 100% certain if my approach is correct, but I’m thinking that if I can get the logit values of each generated token and average them, I could get the overall confidence score of the generated sequence.

Would anybody know how I could do this? Thanks.

You can use the arguments output_scores=True, return_dict_in_generate=True in generate

Hmm neither option seems to work. Is outputs = model.generate(tokens['input_ids'], output_scores=True) the correct syntax?

Apparently you have to set both arguments to True. I was only setting one. My mistake misreading the answer.