Cannot Setup Mixtral Models and Other Models on Inference Endpoints

Inference Endpoints on the Hub
2

Having the same problem. I keep getting sharding errors like these issues:

  1. cognitivecomputations/dolphin-2.5-mixtral-8x7b · HuggingFace Inference Endpoints Issue (Detailed Information)
  2. tiiuae/falcon-7b-instruct · Trying to deploy this using Inference Endpoints