How can I view the output of the answer?

Beginners
1

I have written this!!

from transformers import LongformerTokenizer, LongformerForMultipleChoice
import torch

tokenizer = LongformerTokenizer.from_pretrained(‘allenai/longformer-base-4096’)
model = LongformerForMultipleChoice.from_pretrained(‘allenai/longformer-base-4096’)

prompt = “In Italy, pizza served in formal settings, such as at a restaurant, is presented unsliced.”
choice0 = “It is eaten with a fork and a knife.”
choice1 = “It is eaten while held in the hand.”
labels = torch.tensor(0).unsqueeze(0) # choice0 is correct (according to Wikipedia ;)), batch size 1

encoding = tokenizer([[prompt, prompt], [choice0, choice1]], return_tensors=‘pt’, padding=True)
outputs = model(**{k: v.unsqueeze(0) for k,v in encoding.items()}, labels=labels) # batch size is 1

the linear classifier still needs to be trained

loss = outputs.loss
logits = outputs.logits

But there is only this!!

Some weights of the model checkpoint at allenai/longformer-base-4096 were not used when initializing LongformerForMultipleChoice: [‘lm_head.bias’, ‘lm_head.layer_norm.weight’, ‘lm_head.decoder.weight’, ‘lm_head.dense.weight’, ‘lm_head.layer_norm.bias’, ‘lm_head.dense.bias’]

  • This IS expected if you are initializing LongformerForMultipleChoice from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
  • This IS NOT expected if you are initializing LongformerForMultipleChoice from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
    Some weights of LongformerForMultipleChoice were not initialized from the model checkpoint at allenai/longformer-base-4096 and are newly initialized: [‘classifier.bias’, ‘classifier.weight’]
    You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.

How I can fix it to show me an output??