Longformer for sequenceclassification

🤗Transformers
1

I am using the Longformer for sequence classification task (taking example code from here), the model is running somehow on a small batch, the issue is with the wrong predictions, is there any example of the Longformer sequence classification for GLUE tasks.
thanks

2

try the example script here

1 Like
3

thanks, so instead of --model_name_or_path bert-base-multilingual-uncased , I could use longformer?

4

Yes, all you’ll need to do is change the model name!

1 Like
5

thanks @valhalla appreciate yours response everytime

6

can you check this notebook quickly, where am I going wrong, the predictions are too poor, I just take a sample of dataset during fine-tuning because of performance issue