I have been trying to obtain the logit values for each token output as follows. However, the output that I get is a vector with -inf in all positions except at the position of the predicted token. This is observed when generation_config.do_sample = True. When generation_config.do_sample = False, the…

-inf values for logit score outputs with model.generate

mbismay May 2, 2024, 7:28pm 2

Same issue! Could anyone please explain how to tackle this?

Topic		Replies	Views
How can I obtain the logits via model.generate()? 🤗Transformers	2	3787	October 8, 2024
Can I get logits for each sequence I acqired from model.generate()? Beginners	1	1375	November 27, 2020
Obtain raw logits before decoding scaling is applied 🤗Transformers	1	23	December 5, 2025
How to get the logits for the T5 model when using the `generate` method for inference? Beginners	3	5227	April 18, 2023
[Announcement] Generation: Get probabilities for generated output 🤗Transformers	63	41789	January 20, 2025