Trainer class optimization for transformer models

[jeffreyyu0602]

I am following the “fine-tuning a pretrained model” on the huggingface transformers tutorial. I replaced the bert-base-uncased model with google/mobilebert-uncased. I can get decent result using a huggingface Trainer, but when I trained using native PyTorch, the accuracy stucks around 50%. I am wondering is the huggingface Trainer doing some optimization that I must explicitly add in the native PyTorch code in order to work? Thank you!