You can use this model memory usage calculator for a general idea: Model Memory Utility - a Hugging Face Space by hf-accelerate
If it’s failing right at the beginning of calling .train() then I don’t think it’s the optimizer RAM as the culprit.