Hi, I want to train Trainer scripts on single-node, multi-GPU setting. Do I need to launch HF with a torch launcher (torch.distributed, torchX, torchrun, Ray Train, PTL etc) or can the HF Trainer alone use multiple GPUs without being launched by a third-party distributed launcher?

How to run single-node, multi-GPU training with HF Trainer?

brando August 17, 2022, 3:38pm 3

4 Likes

Topic		Replies	Views
Which method is use HF Trainer with multiple GPU? 🤗Transformers	4	1220	June 19, 2023
Distributed Training w/ Trainer 🤗Transformers	8	6249	May 31, 2022
Does the HF Trainer class support multi-node training? Beginners	4	2117	January 9, 2023
Single Node Multi GPU FlanT5 fine-tuning using HF Dataset and HF Trainer 🤗Transformers	4	1737	July 5, 2023
Training using multiple GPUs Beginners	20	18060	February 25, 2024