Hi,

I am trying to calculate the perplexity from the `generate`

function. I use beam search as the decoding strategy, but I would like to get the perplexity for all outputs of the third sentence (or maybe other, not the first one).

To calculate the perplexity, I need first calculate the loss, but I didnâ€™t find a way to extract the logits from the `generate`

function with beam search. I found that the `scores`

are the â€śBeam transition scores for each vocabulary token at each generation step. Beam transition scores consisting of log probabilities of tokens conditioned on log softmax of previously generated tokens in this beam.â€ť According to this post: [Announcement] GenerationOutputs: Scores, Attentions and Hidden States now available as outputs to generate, `scores`

now correspond to all processed lm head logits + the current beam_scores for each output token. So I am confused how can I extract the logits to calculate the loss or calculate the perplexity directly from `generate`

function.