I am trying to train a GPT-2 large language model using Deepspeed plugin of accelerate. Every time when the script calls this line, loss = model(torch.stack(batch['input_ids']).T,labels=torch.stack(batch['input_ids']).T).loss , I get the following error: RuntimeError: Expected tensor for argument …

Getting torch.cuda.halfTensor error while using DeepSpeed with accelerate

madscientist1224 May 18, 2023, 7:38pm 6

I solved by changing accelerate config to not use fp16 (stage 2)

Topic		Replies	Views
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! 🤗Accelerate	1	771	May 31, 2024
Issues when using `accelerate` with `fp16` Intermediate	4	12097	January 22, 2024
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:2 and cuda:0! (when checking argument for argument index in method wrapper_CUDA__index_select) DeepSpeed	5	3474	August 26, 2024
How to use trainer with deepspeed Beginners	0	342	January 12, 2024
Using Accelerate with DeepSpeed for WNUT Example 🤗Accelerate	1	865	July 19, 2023