I am trying to train a GPT-2 large language model using Deepspeed plugin of accelerate. Every time when the script calls this line, loss = model(torch.stack(batch['input_ids']).T,labels=torch.stack(batch['input_ids']).T).loss , I get the following error: RuntimeError: Expected tensor for argument …

Getting torch.cuda.halfTensor error while using DeepSpeed with accelerate

madscientist1224 May 18, 2023, 7:38pm 6

I solved by changing accelerate config to not use fp16 (stage 2)

Topic		Replies	Views
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! 🤗Accelerate	1	512	May 31, 2024
RuntimeError: tensors must be contiguous when finetuning GPT-J-6B using PEFT Lora DeepSpeed	0	794	July 29, 2023
Error 'expected scalar type Half but found Float' 🧨 Diffusers	2	3585	November 8, 2022
RuntimeError: input type (torch.FloatTensor) and weight type (torch.HalfTensor) should be the same or input should be a MKLDNN tensor and weight is a dense-tensor 🤗Datasets	0	684	January 3, 2023
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.LongTensor [1, 128]] is at version 3; expected version 2 instead. Hint: the backtrace further above shows the operation that failed t 🤗Transformers	1	1684	August 16, 2024