Hi, I am finetuning a BARTForConditionalGeneration model. I am using Trainer from the library to train so I do not use anything fancy. I have 2 gpus I can even fit batch size 8 or 16 during training but after first epoch, I always receive Cuda Out of memory error. I find it strange because my evalu…

Cuda out of memory during evaluation but training is fine

vpetren October 31, 2020, 5:42pm 6

Thanks, switched over to finetune_trainer.pyand it works great now.

Topic		Replies	Views
CUDA out of memory when using Trainer with compute_metrics 🤗Transformers	20	30979	October 5, 2023
CUDA out of memory only during validation not training 🤗Transformers	3	3795	May 9, 2023
CUDA out of memory error while predicting (evaluation) 🤗Transformers	1	505	March 22, 2024
torch.cuda.OutOfMemoryError when evaluate while traning 🤗Transformers	0	389	October 8, 2023
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 256.00 MiB (GPU 0; 39.56 GiB total capacity; 37.84 GiB already allocated; 242.56 MiB free; 37.96 GiB reserved in total by PyTorch) 🤗Transformers	2	3856	June 7, 2023