I’m finetuning GPT2 on my corpus for text generation. I am also using the Trainer class to handle the training. I have multiple gpu available to me. As I understand from the documentation and forum, if I wanted to utilze these multiple gpu for training in Trainer, I would set the no_cuda parameter t…

Finetuning GPT2 using Multiple GPU and Trainer

valhalla September 18, 2020, 9:26am 6

Could you post the command/script_snippet that you used to launch training ?

Topic		Replies	Views
Training using multiple GPUs Beginners	20	20107	February 25, 2024
Custom model with two pretrained models fails multi GPU training when using the Trainer 🤗Transformers	0	245	March 2, 2023
How can I use trainer.train() in runpod's multi gpu? 🤗Transformers	0	421	August 4, 2023
Finetuning GPT2 with user defined loss Beginners	56	16107	July 23, 2023
Fine-tunning llama2 with multiple GPU hugging face trainer 🤗Transformers	8	3385	March 7, 2024