RuntimeError: Failed to generate response from together API: 429 Client Error: Too Many Requests for url: https://huggingface.co/api/models/meta-llama/Meta-Llama-3.1-8B-Instruct?expand=inferenceProviderMapping
Tried using Inference Provider (Together AI as provider) via InferenceClient
First time usage. I have never used this service before
Get the above error
I have paid for Pro Plan ($9/month)
I have also obtained access for Meta’s llama models which require permission
I have also added headers = {“X-wait-for-model”: “true”} when initiating the InferenceClient()
Yet I still can’t use at all?!?! Where does this rate limit come from?
What is even the point of paying for pro plan if I can’t use these models at all??
1 Like
I don’t know if it’s a problem with the Inference Provider or a lingering effect from the previous server malfunction…
Either way, it’s a problem with the paid service… @meganariley @michellehbn
@John6666 Are you part of HuggingFace team?
I went to check Together AI’s page for their list of models and realized that their version of Lllama3.1 8B is meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo
Is this the reason why my InferenceClient cannot pull model info from huggingface using this link https://huggingface.co/api/models/meta-llama/Llama-3.1-8B-Instruct?expand=inferenceProviderMapping?
I have also checked the json object returned from this link. Turns out there is no Together AI among the providers. Is this the root cause for 429 error?
1 Like
Including HF Discord, forums and hubs are basically based on mutual assistance between users, and it is common for IT companies to have staff who are sometimes present and sometimes absent. Whether that is a good thing or not is another question…
Is this the reason why
Even if the names are different, it seems that the assignments themselves have been made. Perhaps only that provider has not been assigned yet…?
