Hello,
I got a question regarding fine tuning a BERT Sentiment model. I am working with Customer Feedback data and want to fine tune “nlptown/bert-base-multilingual-uncased-sentiment” for my specific usecase as there are always patterns for wrong classified sentiments no matter which pretrained model I use. However, when I fine tune it I get weird losses like this:
These loss jumparounds appear in the same pattern no matter if I reduce the learning rate, choose different batch sizes or update my weight decay. I am working with a Trainer() insance. Nevertheless, if I run the new trained model with test data the results are looking fine. Now finally my question: Can I still use this model? The losses indicate overfitting, don’t they?