Text generation pipeline - output_scores parameter

[valhalla]

the text-generation pipeline doesn’t return scores, however you could the generate method directly, to get the scores, this should help

Generation Probabilities: How to compute probabilities of output scores for GPT2

Now that it is possible to return the logits generated at each step, one might wonder how to compute the probabilities for each generated sequence accordingly. The following code snippet showcases how to do so for generation with do_sample=True for GPT2: import torch from transformers import AutoModelForCausalLM from transformers import AutoTokenizer gpt2 = AutoModelForCausalLM.from_pretrained("gpt2", return_dict_in_generate=True) tokenizer = AutoTokenizer.from_pretrained("gpt2") input_ids …