T5ForConditionalGeneration, How to get prediction probabilities or logits at the inference time? (to calculate perplexity)

🤗Transformers
1

Hello. It is the situation exactly as described in the title.
I want to get model’s prediction probabilities or logits to calculate perplexity.
But as you know we can not get prediction probabilities if we use model.generate() method built in T5ForConditionalGeneration—we only can get prediction tokens, not probabilities.

One possible method I think is to use model() method (forward). Because it returns logits and I can use those logits to calculate perplexity. But the additional unnecessary coding for decode logits should be followed. So I wanna know if there is any better solutions to calculate perplexity.

So my question is
How to get prediction probabilities or logits at the inference time?
or
Teach me if there is other better way to calculate perplexity!

Thanks a lot in advance :slight_smile:

2 Likes