I’m trying to fine-tune Llama-2-13b-chat for machine translation with lightning trainer and deepspeed. The test reported here is on 4 rtx 3090 with an 8bit quantized version of the model and LoRa and I get this error. This is the code used for training trainer = L.Trainer( devices=4, acc…

RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:2 and cuda:0! (when checking argument for argument index in method wrapper_CUDA__index_select)

Kason123 January 12, 2024, 4:07pm 4

Did you figure out a workaround for this issue?

1 Like

Topic		Replies	Views
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:1 and cuda:0! 🤗Transformers	28	114871	November 17, 2024
Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cuda:1! 🤗Accelerate	1	892	September 20, 2023
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! 🤗Accelerate	1	771	May 31, 2024
Tensor not on the same device after using BitsandBytesConfig 🤗Transformers	0	106	June 27, 2024
How to use trainer with deepspeed Beginners	0	342	January 12, 2024