HF Agents Course 404 Client Error: Not Found for url

🤗Hub
1

Hey guys

I’m struggling with this error:

404 Client Error: Not Found for url: https://router.huggingface.co/hf-inference/models/Qwen/Qwen2.5-Coder-32B-Instruct/v1/chat/completions

The code is taken from here:

It’s appearing with any instruct model i tried (including those with special access such as Llama models)

What’s that?

Would be grateful for any help

I saw there is maybe a problem with zero-scale or something like that, but i used popular models, I’m not sure that this is a reason

1 Like
2

I think this is due to a large number of models whose deployment has been canceled, as well as major changes to the library used for the Inference API. I’m not familiar with the workaround for this issue on LlamaIndex, but according to GitHub, updating the HF library should still make it work.

To update hf_hub library

pip install -U huggingface_hub
3

Hi, thanks for your answer!
Unfortunately updating didn’t help, I’ve tried it

1 Like
4

Hmm, in that case, do you need to update LlamaIndex, or has it become unusable due to further specification changes…?
I think the model itself is deployed via Inference Provider.

However, if you are not particularly attached to that model, it might be better to look for an alternative. More detailed information is available in the Agents course channel on Hugging Face Discord.

Alternative API Endpoints / local models for smolagents

5

Everything is up-to-date

Actually I’m using some other models directly, but just want to cope with that problem. Maybe someone knows how to fix it

Thank you anyway

1 Like
6

https://router.huggingface.co/hf-inference/models/Qwen/Qwen2.5-Coder-32B-Instruct/v1/chat/completions

hf-inference <= this

I see. Let me explain the situation. It is normal for this URL not to work because this model has not been deployed with HF Inference. Currently, very few LLMs are deployed via HF Inference. Most are deployed via other Inference Providers.

If LlamaIndex does not have a feature to switch the Inference Provider or set it to ="auto", only few models will work.

1 Like
7

Yes, I think you’re right and the problem is in the framework or so. Just don’t understand why they put this example in the course.
Actually it must be available for deploy with HF Inference, because there is a code for deploying:

import os
from huggingface_hub import InferenceClient

client = InferenceClient(
    provider="auto",
    api_key=os.environ["HF_TOKEN"],
)

completion = client.chat.completions.create(
    model="Qwen/Qwen2.5-Coder-32B-Instruct",
    messages=[
        {
            "role": "user",
            "content": "What is the capital of France?"
        }
    ],
)

print(completion.choices[0].message)

But maybe this is the only way to deploy it, and HuggingFaceInferenceAPI is restricted now (despite this code is in the course).

1 Like
8

Just don’t understand why they put this example in the course.

Yeah. When the course was created, that method was available…
If it’s just a matter of library versions or so, we can just stick with the old ones, but for the “Agents” course, we need as many examples as possible of using “external APIs,” whether provided by HF or a third party…

But AI services change a lot in just a few months. It’s difficult to keep them up to date.

1 Like
9

Agree. But it can be easily resolved at least with linked discussions about problems&solutions on this forum for instance. Just one button on the page “Got stuck, but found a solution? Tell us more” or so. I saw the same on the other platform. Or just a little checklist, like..there are may appear some problems. Check you have Pro status to use HF Inference API, check deploy button etc etc

Without claims to authors, always there are ways to make a course better

Thanks for you help!

1 Like