Can't run Janus with HuggingFaceEndpoint

Intermediate
1

I’m trying to integrate huggingface endpoint with LangChain project using HuggingFaceEndPoint proviede by LangChain.

llm = HuggingFaceEndpoint(
    repo_id="deepseek-ai/Janus-Pro-7B",
    task="image-text-to-text",
    max_new_tokens=512,
    do_sample=False,
    repetition_penalty=1.03
)

But it returns with 500 Server Error, and unknown variant ‘any-to-any’. I’m using task like ‘image-text-to-text’ and ‘image-to-text’ and they have same return value.

image
image1943×190 15 KB

Not only Janus-Pro-7B can’t run, other VLM like Qwen2.5-VL-7B-Instruct and glm-4v-9b can’t run. I suspect this HuggingFaceEndpoint could not run VLM.

1 Like
2

Oh…

This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API: The HF Inference API does not support any-to-any models for transformers library.

3

Thanks! What VLM can I use with HF inference API?

1 Like
4

This?

5

This? Qwen/Qwen2-VL-7B-Instruct

6

Sorry!This model return thishuggingface_hub.errors.HfHubHTTPError: 503 Server Error: Service Temporarily Unavailable for url: https://router.huggingface.co/hf-inference/models/Qwen/Qwen2.5-VL-7B-Instruct/v1/chat/completions
And I can exclude network reason because I can run microsoft/Phi-3-mini-4k-instruct

1 Like