How to get log probs if we already have a generation?

Beginners
1

Hi! I’m wondering if there’s any way I could get log probs for tokens in a generated sequence if I already have a generation. I was going through this post about getting log probs with generate().

However, if I already have a sequence that was generated by the model using a specific decoding method (greedy, top-k) and I want to get log probs for each token in that generated sequence, is that possible?

2

You could try just feeding your sequence into the model all at once with a single forward call. If you need to simulate the effects of top-k, etc you could then run the logits processors on the logits you get back from the forward call

1 Like