Hi, I am trying to use ‘ Accelerate ’ for running inference on a PMC-Llama model using a GPU and CPU. Here is my code, import transformers from accelerate import init_empty_weights from transformers import AutoConfig, AutoModelForCausalLM config = AutoConfig.from_pretrained("chaoyi-wu/PMC_LLAMA_7B…

@abhibha If you are sure that the warning was triggered by the presence of model.tie_weights(), and the warning disappears when that line is removed, then it’s likely that the warning is related to the usage of model.tie_weights() in your specific context. Without knowing the specific code and the…

Error: The model weights are not tied. Please use the `tie_weights` method before using the `infer_auto_device` function, even after adding model.tie_weights()

Beginners

abhibha July 14, 2023, 9:13pm 5

@Falah Thanks! I am already using transformers version - 4.30.2 . And still am getting the error.

Topic		Replies	Views
Meta device error while instantiating model 🤗Accelerate	5	7223	April 1, 2025
How to properly wrap a model for training with accelerate? 🤗Accelerate	1	1353	September 20, 2023
Multi-gpu inference llama-3.2 vision with QLoRA 🤗Accelerate	4	185	April 25, 2025
Does pipline with accelerate use "with init_empty_weights():"? 🤗Accelerate	3	247	April 15, 2024
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:1 and cuda:0! 🤗Transformers	28	116007	November 17, 2024

Error: The model weights are not tied. Please use the `tie_weights` method before using the `infer_auto_device` function, even after adding model.tie_weights()

Related topics