During inference time, is there a way to extract the top k tokens by probabilities for the first token generated by T5 ?
I have fine-tuned T5 model using the T5ForConditionalGeneration method and now I’m calling a forward call:
output = model(input_ids=source_ids,attention_mask=source_masks,return_dict=True)
But it raises an error:
ValueError: You have to specify either decoder_input_ids or decoder_inputs_embeds
In the T5 training documentation, it says that the forward call " automatically creates the correct decoder_input_ids" which seems that it is not true in my case. How do I handle that forward call?