Run vllm docker on space

Spaces
1

I just try to run vLLM docker with space,
but there is something weird.

I can call https://xxx.hf.space/v1/models
But I got error when call https://xxx.hf.space/v1/chat/completions
the error was
“Server unavailable, error code: 349453”

I try to find the error on google but found nothing.
Thanks

1 Like
2

hi @sofianhw,

Could please share a Space with Dockerfile example?

3

I just add prefix,
and now it works

4

I have the same error.

What do you mean by ‘I just add prefix’?

Can you give an example?

Thanx for your help

5

I just copy all files in these folder:

and change everything route in api_server.py that start (“/v1/xxx”) to (“/api/v1/xxx”).
and just run the python api_server.py with arguments.

6

Tx for your help ! :+1:

I’ve found another way:
Add just ‘‘extra_query={‘id’: time.time_ns()}’’ on call params for client.chat.completions.create (with openai client lib 1.9.0)

1 Like
7

Cool,
I will try also :+1:

8

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.