I’ve been trying to figure out the nature of the deepspeed integration, especially with respect to huggingface accelerate. It seems that the trainer uses accelerate to facilitate deepspeed. But when I look at the documentation, it seems that we still use deepspeed as the launcher, or the pytorch di…

It seems that the trainer uses accelerate to facilitate deepspeed. But when I look at the documentation, it seems that we still use deepspeed as the launcher, or the pytorch distribute deepspeed --num_gpus=2 your_program.py <normal cl args> --deepspeed ds_config.json or python -m torch.distribut…

Using deepspeed script launcher vs accelerate script launcher for TRL

🤗Accelerate

marcsun13 January 24, 2024, 8:34pm 4

Yes, you can see it in this line here. Also see pytorch doc.

2 Likes

Topic		Replies	Views
Deepspeed script launcher vs accelerate script launcher for TRL DeepSpeed	0	370	December 25, 2023
Exact difference between Transformers' and Accelerate's DeepSpeed integrations? DeepSpeed	5	828	February 13, 2024
Difference between using the Trainer class vs Accelerate library DeepSpeed	0	910	June 27, 2023
Besides writing your own training loop, is there any other advantage for using it with deepspeed? 🤗Accelerate	2	589	July 4, 2023
Best practice to run DeepSpeed DeepSpeed	2	1563	December 25, 2023

Using deepspeed script launcher vs accelerate script launcher for TRL

Related topics