OpenAIGPTLMHeadModel: How to adapt example code snippet to compute probability of a sentence?


I want to use GPT to compute the probability of a sentence. I currently use the code snippet given here:

I currently use the following code (Ilustrated for the example sequence on the site):

import torch
from transformers import OpenAIGPTTokenizer, OpenAIGPTLMHeadModel
import math

tokenizer = OpenAIGPTTokenizer.from_pretrained('openai-gpt')
model = OpenAIGPTLMHeadModel.from_pretrained('openai-gpt', return_dict=True)

sequence = "Hello, my dog is cute"
inputs = tokenizer(sequence, return_tensors="pt")
outputs = model(**inputs, labels=inputs["input_ids"])
loss = outputs.loss
logits = outputs.logits
# untill here it's just the code snippet from the documentation 

# apply softmax to logits 
softmax_logits = torch.softmax(logits[0],1)
# get token indices 
token_indices = tokenizer.encode(sequence)
scores_for_each_token = []
for i, index in enumerate(token_indices,0): 
    individual_score_for_token = softmax_logits[i][index].item()
#compute the sentiment score as the product of each token 
final_score =

Does this seem like a reasonable approach?

Thanks in advance :)!