Train loss is decreasing, but accuracy remain the same

[sgugger]

This means you are overfitting (training loss diminished but no improvement in validation loss/accuracy) so you should try using any technique that helps reduce overfitting: weight decay, more dropout, data augmentation (if applicable)…