03 Error When Using Qwen/Qwen2.5-VL-32B-Instruct with Inference Provider

IEAINC1 · September 30, 2025, 12:47am

I’m currently using the Hugging Face Inference Provider to run the model Qwen/Qwen2.5-VL-32B-Instruct, but I keep receiving a 503 Service Unavailable error when making inference requests.

Model: Qwen/Qwen2.5-VL-32B-Instruct
Endpoint: (please add your endpoint URL if applicable)
Error: 503 Service Unavailable
Request type: (e.g., text generation / image understanding — specify if needed)
Frequency: (e.g., happens every request / intermittent)

Could you please help me understand why this error occurs and how to resolve it? Is there any current issue with the Inference Provider or with this specific model?

John6666 · September 30, 2025, 1:19am

Hmm… It works…

from huggingface_hub import InferenceClient

client = InferenceClient(
    model="Qwen/Qwen2.5-VL-32B-Instruct",
    provider="fireworks-ai",
    api_key="hf_********",
)

IMAGE_URL = "https://upload.wikimedia.org/wikipedia/commons/4/4d/Cat_November_2010-1a.jpg"

resp = client.chat.completions.create(
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "Describe the image in one sentence."},
            {"type": "image_url", "image_url": {"url": IMAGE_URL}},
        ],
    }],
    max_tokens=128,
)

print(resp.choices[0].message["content"]) # 这是一张猫的照片。图片中的猫有条纹的毛色，主要是棕色和黑色相间的条纹。它的耳朵竖立，眼睛是绿色的，正直视着镜头。猫的胡须清晰可见，前爪放在一个浅色的表面上，可能是石头或混凝土。背景模糊，显示出一些树枝和天空，表明猫可能在户外。整体画面给人一种宁静而专注的感觉，猫的姿态显得警觉且好奇。

IEAINC1 · September 30, 2025, 4:23am

When I send a simple input to the model, it works fine, but when I include a system prompt, the response just comes back as weird characters like \t\n\t\n\t\n\t\n instead of proper text.

requests.exceptions.HTTPError: 503 Server Error: Service Temporarily Unavailable for url: https://router.huggingface.co/fireworks-ai/inference/v1/chat/completions

The above exception was the direct cause of the following exception:

{
    "choices": [
        {
            "finish_reason": "length",
            "index": 0,
            "message": {
                "role": "assistant",
                "content": "\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n\t\t\\n\n",
                "tool_call_id": null,
                "tool_calls": null
            },
            "logprobs": null
        }
    ],
    "created": ##################,
    "id": "##################",
    "model": "accounts/fireworks/models/qwen2p5-vl-32b-instruct",
    "system_fingerprint": null,
    "usage": {
        "completion_tokens": 2048,
        "prompt_tokens": 1140,
        "total_tokens": 3188
    }
}

John6666 · September 30, 2025, 7:31am

In that case, I suspect it’s likely an issue on the Endpoint side, but is this error-out intended behavior? @michellehbn

meganariley · October 2, 2025, 2:06pm

Hey @IEAINC1 thanks for posting! Are you still seeing this error message? Let us know!

Topic		Replies	Views
Inference API Issues Beginners	0	2722	September 9, 2021
Qwen failed to build on Huggingface Inference Endpoint Models	0	238	February 23, 2024
Help using inference endpoint with Llama 3.1 405B Instruct Inference Endpoints on the Hub	1	176	August 30, 2024
All Hosted Inference Api's are giving the http 422 error Beginners	0	303	July 17, 2023
Constant 503 error for several days when running LLAMA 3.1 Inference Endpoints on the Hub	5	405	April 25, 2025

03 Error When Using Qwen/Qwen2.5-VL-32B-Instruct with Inference Provider

Related topics