As far as I know, for Pytorch, RandomSampler can not be directly used in the distributed data parallel training since DistributedSampler is desired (this link discusses the problem). I am wondering whether accelerator.prepare(dataloader) handles the data split for multiple GPUs if I use the RandomSa…

Can accelerator handle the distributed sampler?

ezio98 December 21, 2021, 1:47am 3

That’s great! Thanks!

Topic		Replies	Views
Using DistributedSampler with accelerate 🤗Transformers	4	352	April 2, 2025
DistributedSampler with Accelerate 🤗Accelerate	1	51	June 10, 2025
Accelerate - WeightedRandomSampler Dataloader Intermediate	1	279	June 18, 2024
Accelerator .prepare() replaces custom DataLoader Sampler 🤗Accelerate	5	1364	March 9, 2025
How does Accelerate ensure uniqueness of data samples across GPUs? 🤗Accelerate	2	902	June 21, 2023