The model mistralai/Mistral-7B-Instruct-v0.1 is too large to be loaded automatically (14GB > 10GB)

Models
1

I was using the model with the inference api , Initially it was working fine but after 2-3 hours in the evening i started to get this error.
Hoe can i resolve and use the model with free Access token.

2

The same Happened with Qwen 2.5 VL 7B and i think they have removed it from hugging face serverless service via hugging face Inference API , try different provider for the API call or use a dedicated server (this worked for me).

3

I think this will only be fixed if someone applies for it.

mistralinference
mistralinference824×578 36.6 KB