Hi,
I really recommend this post: A Recipe for Training Neural Networks to debug your training run. The best tip for me is: take 1 training example, and see whether the model is able to overfit it (i.e. achieve 100% accuracy). If not, then there’s a bug in your model.
See also this guide which we wrote for debugging your training pipeline with the Trainer class: Debugging the training pipeline - Hugging Face NLP Course.