How to obtain GPT2 output after softmax layer, along with the gradient information?

🤗Transformers
1

Hello,

I know how to obtain the classification scores of the GPT2DoubleHeadsModel before the softmax layer:

model_gpt2 = GPT2DoubleHeadsModel.from_pretrained('gpt2-medium', output_hidden_states = True)

# retrieve the classification scores before the softmax
classification_scores = model_gpt2(input_ids=input_ids, mc_labels=mc_labels)[2]

How can I obtain the classification scores of the GPT2DoubleHeadsModel after the softmax layer, along with the gradient information?

Thank you,