I bought huggingface pro. I am trying to use the model microsoft/Phi-3-mini-128k-instruct on HuggingFace’s Serverless API since yesterday through the following code:
response = self.client.chat.completions.create(
model=self.model_name,
messages=[{“role”: “user”, “content”: prompt}],
temperature=0,
max_tokens=300
)
However, I keep having the same errors:
and
504 Server Error: Gateway Time-out for url: https://api-inference.huggingface.co/models/microsoft/Phi-3-mini-128k-instruct/v1/chat/completions
I tried other models such as meta-llama/Llama-3.2-1B and I keep having the same problem. In the midst of the errors, there were like one or two sucessful requests. Is this normal?