I have trained a ViTAdapter model, that I added a prediction head to, with the following code:
model = AutoAdapterModel.from_pretrained( 'google/vit-base-patch16-224-in21k', from_tf=bool(".ckpt" in 'google/vit-base-patch16-224-in21k'), config=config, cache_dir=None, revision='main', use_auth_token=None, ignore_mismatched_sizes=False, ) model.add_image_classification_head( "classification_head", num_labels = len(labels), id2label=id2label, )
I now want to load up the weights, so I can use the model on new images. However, when I call the following:
model = AutoAdapterModel.from_pretrained("path_to_model_weights")
I get the error
Some weights of the model checkpoint at vit_adapter_orange_surgical_task/vit-adapter-surgery were not used when initializing ViTAdapterModel
I believe this is because the weights from the added prediction head cannot be initialized onto a basic ViTAdapterModel. Is there a way to call .from_pretrained from a custom model instead of AutoAdapterModel? Alternatively, could a prediction head be added onto the loaded model above and filled with the weights from the trained head?
I appreciate any help and advice!