Fine-tuning Segment Anything Model: Call up a saved model

🤗Transformers
1

Hi all,

I have successfully trained my SAM model with the good guidance from the following blog post.

Blog Post: Fine tune Segment Anything (SAM) for images with multiple masks

I saved the weights of the trained model using the following code:

torch.save(model.state_dict(), ‘model_weights.pth’).

However, I am now faced with the question of how to use this saved model for my code. The original version of SAM is loaded via the following code:

from segment_anything import SamAutomaticMaskGenerator, sam_model_registry

sam = sam_model_registry(vit_h)(checkpoint=“/content/drive/MyDrive/…/SAMFineTune/sam_vit_h_4b8939.pth”)

Note: I had to replace the square brackets at vit_h with round ones to post it here.

Can anyone help me how to use the trained model now in my code to create masks? Unfortunately, I haven’t found anything yet to solve the problem.

1 Like
2

.(torch.load(model_save_path)) is the function to load it. Forward passes should be weighed according to your trained model now, in case this helps anybody!

3

Hello! I have the same issue with my fine-tuned weights from SAM.
@Ghouloud I can load the weights thanks to your tip but I can’t create the sam generator.
@anon69151022 did you find any solution ?

1 Like
4

Hello!

By SAM generator are you referring to the dataloader?

5

No, it’s about SamAutomaticMaskGenerator() function, as explained in Blog Post: Fine tune Segment Anything (SAM) for images with multiple masks .

I can’t run the following lines :
sam = sam_model_registry(MODEL_TYPE)(checkpoint=CHECKPOINT_PATH).to(DEVICE)
mask_generator = SamAutomaticMaskGenerator(sam)