Always getting RuntimeError: CUDA out of memory with Trainer

You can use this model memory usage calculator for a general idea: Model Memory Utility - a Hugging Face Space by hf-accelerate

If it’s failing right at the beginning of calling .train() then I don’t think it’s the optimizer RAM as the culprit.