T5 transformer tokens and scores

Beginners
1

Hello,

I am using beam search with the pretrained T5ForConditionalGeneration model. I am trying to implement some sort of uncertainty estimation at a token level. Therefore, I was looking at the ‘scores’ by setting return_dict_in_generate=True and output_scores=True.

output=self.model.module.model.generate(input_ids=input['input_ids'],
                                            attention_mask=input['attention_mask'], 
                                            num_beams=8,
                                            return_dict_in_generate=True,output_scores=True,
                                            output_hidden_states=True,output_attentions=True,
                                            early_stopping=True, max_length=200)

Now, output['scores'] returns a tuple as instructed by the docs. What does each tuple mean? Each tuple is a tensor of size tensor of shape (batch_size*num_beams*num_return_sequences, config.vocab_size). I can’t seem to visualize what each tuple represent. Any help would be greatly appreciated.