Run vllm docker on space

I just try to run vLLM docker with space,
but there is something weird.

I can call
But I got error when call
the error was
“Server unavailable, error code: 349453”

I try to find the error on google but found nothing.

hi @sofianhw,

Could please share a Space with Dockerfile example?

I just add prefix,
and now it works

I have the same error.

What do you mean by ‘I just add prefix’?

Can you give an example?

Thanx for your help

I just copy all files in these folder:

and change everything route in that start (“/v1/xxx”) to (“/api/v1/xxx”).
and just run the python with arguments.

Tx for your help ! :+1:

I’ve found another way:
Add just ‘‘extra_query={‘id’: time.time_ns()}’’ on call params for (with openai client lib 1.9.0)

I will try also :+1:

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.