How to use RMSE loss for regression

Hi, I’m fine-tuning a transformer (ForSequenceClassification variant) as a regression task, but I want to use RMSE instead of MSE as the loss function, because the paper I am trying to replicate does so. How can I do this?

Thanks