Hi @caput maybe these two links help: gpt_sent_prob Generation Probabilities: How to compute probabilities of output scores for GPT2