CUDA out of memory when using Trainer with compute_metrics

[Randool]

Yes, and I have found that compute_metrics will save all the logits before move them from CUDA to memory, just as sgugger said:

@sgugger

If your dataset is large (or your model outputs large predictions) you can use eval_accumulation_steps to set a number of steps after which your predictions are sent back to the CPU (slower but uses less device memory).

Sorry for forgetting mark this question as SOLVED