I’m trying to fine-tune Llama-2-13b-chat for machine translation with lightning trainer and deepspeed. The test reported here is on 4 rtx 3090 with an 8bit quantized version of the model and LoRa and I get this error. This is the code used for training trainer = L.Trainer( devices=4, acc…

RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:2 and cuda:0! (when checking argument for argument index in method wrapper_CUDA__index_select)

Kason123 January 12, 2024, 4:07pm 4

Did you figure out a workaround for this issue?

1 Like

Topic		Replies	Views
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! 🤗Accelerate	1	496	May 31, 2024
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! I am on a single T4 GPU 🤗Accelerate	6	637	June 10, 2024
Trainer.train throws RuntimeError: Expected all tensors to be on the same device Beginners	5	2907	May 17, 2023
How to use trainer with deepspeed Beginners	0	291	January 12, 2024
GPU error on LoRA for token classification 🤗Transformers	2	652	June 19, 2023