I am trying to fine-tune the “distilbert-base-uncased” with the “emotions” dataset (6 labels).
I am using DistilBertTokenizer.
When I use DistilBertForSequenceClassification.from_pretrained()
I get an accuracy/f1 of only 35%,
but when II use AutoModelForSequenceClassificationfrom_pretrained()
I get an accuracy/f1 over 90%
I thought it’s better to use a specific model than the Auto ones but it seems Auto works much better, what could be a logical explanation for this?
Thanks.