Quantizing Facebook's segment anything model

I am trying to quantize Facebook’s segment anything transformer model “facebook/sam-vit-huge”, especially the encoder side so that I can get a lower inference time. I have an NVIDIA RTX 3060 12GB RAM GPU. Can anyone help me get started with this?

1 Like

I concern too