Hi, After training and saving falcon-7b-instruct, I am attempting to load the model for inference using accelerate init_empty_weights, and load_checkpoint_and_dispatch. When the call to load_checkpoint_and_dispatch() is made, AttributeError: ‘FalconModel’ object has no attribute ‘model’ is returned…

AttributeError: 'FalconModel' object has no attribute 'model'

marcsun13 January 24, 2024, 6:24pm 2

Hi @Borell, can you provide a reproducer. My hunch is that you are not initializing the correct model architecture under init_empty_weights .

Topic		Replies	Views
Meta device error while instantiating model 🤗Accelerate	5	7062	April 1, 2025
Created a pretrained model but does not seem to working Beginners	2	108	July 3, 2024
Fine tuned model gave unreadable responses 🤗Transformers	1	327	June 18, 2024
Accelerate: 'RobertaModel' object has no attribute 'roberta' Intermediate	1	505	August 29, 2023
Does pipline with accelerate use "with init_empty_weights():"? 🤗Accelerate	3	236	April 15, 2024