Hugging Face Forums

Which data parallel does trainer use? DP or DDP?

🤗Transformers

brando August 17, 2022, 3:03pm 3

perhaps useful to you: Using Transformers with DistributedDataParallel — any examples?

3 Likes

How to run an end to end example of distributed data parallel with hugging face's trainer api (ideally on a single node multiple gpus)?

Topic		Replies	Views	Activity
Running a Trainer in DistributedDataParallel mode 🤗Transformers	1	1457	October 24, 2020
Using Transformers with DistributedDataParallel — any examples? Intermediate	11	23594	May 8, 2023
Multi gpu training 🤗Transformers	3	6038	April 24, 2022
Model's evaluation in DDP training is using only one GPU Beginners	1	1068	September 14, 2023
Trainer is not using multiple GPUs in the DP setup Beginners	0	831	April 9, 2023