Hugging Face Forums

Which data parallel does trainer use? DP or DDP?

🤗Transformers

brando August 17, 2022, 3:03pm 3

perhaps useful to you: Using Transformers with DistributedDataParallel — any examples?

3 Likes

How to run an end to end example of distributed data parallel with hugging face's trainer api (ideally on a single node multiple gpus)?

Topic		Replies	Views	Activity
Using Transformers with DistributedDataParallel — any examples? Intermediate	11	23454	May 8, 2023
Multi gpu training 🤗Transformers	3	6027	April 24, 2022
Trainer API for Model Parallelism on Multiple GPUs 🤗Transformers	5	4228	September 10, 2024
Running a Trainer in DistributedDataParallel mode 🤗Transformers	1	1450	October 24, 2020
How to use FSDP + DPP in Trainer 🤗Transformers	1	1016	April 24, 2023