Training using multiple GPUs

aliosk March 22, 2021, 6:22pm 12

@sgugger The model is the routing transformer language model (RoutingTransformerLM). The source code is here:

Topic		Replies	Views
Finetuning GPT2 using Multiple GPU and Trainer 🤗Transformers	14	6791	May 22, 2023
Custom model with two pretrained models fails multi GPU training when using the Trainer 🤗Transformers	0	245	March 2, 2023
Multiple gpu training 🤗Transformers	1	2588	August 10, 2024
Which method is use HF Trainer with multiple GPU? 🤗Transformers	4	1564	June 19, 2023
How to run single-node, multi-GPU training with HF Trainer? 🤗Transformers	5	15224	October 16, 2024