Hi there!
I have deployed a demo for users to use my model. The demo provides several variables for users to choose:
For instance, the “input type” provides three options and the system will do different things for each option. However, I found that if there are multiple users using this demo, e.g. user A and B, if A changes the “input type”, the logic will also be changed for B, even if B doesn’t change its type. This would cause the difference between the option shown for B and the code executed by the demo.
I wonder is there any way to solve this problem? Any suggestions would be appreciated!!!
ps: Here is the link of my demo: Demo_ProTrek_650M_UniRef50 - a Hugging Face Space by westlake-repl
If you configure all components as Gradio standard, like a very plain space, Gradio seems to do so on its own.
(Otherwise it would surely be a terrible in image generation space or something…)
The only thing that would rather be shared is the disk cache and console logs.
Since it seems to be a hybrid of Gradio and Docker, maybe that’s not the way it’s supposed to work?
I think it’s a pretty tough process to do manually on your own.
Thank you for the reply! Yes I deployed this demo in a docker container, and I don’t know whether this is the reason. Maybe I have to do more tests.
Maybe if you port it to pure Gradio it will work the way you want it to.
But I don’t think Gradio is particularly good. And there are a lot of errors and compatibility issues that really disgust me.
I don’t see that many people in the Docker space having those problems, so I wonder if pure Docker without Gradio would work well, too.
I’ve never used Docker Space so it would be nice if others could explain it here.
If it is pure Gradio, the parameters can be easily set as shown below.
At first I used pure Gradio. However pure Gradio only supports to install dependencies through requirement.txt. But I want do more operations beyond that (e.g. many linux commands) so Docker is more suitble for me.
I’m not sure the problem is caused by my incorrect use of Gradio components or I have missed something to correctly provide service to multiple users. Anyway, I will check the guidance link you provide for reference.
If you can use Gradio, it is a quick story.
pure Gradio only supports to install dependencies through requirement.txt.
That’s what I thought at first too, but actually it’s not true. I don’t remember seeing any documentation of any kind, but I imitated what others had done this way.
&|
import subprocess
subprocess.run('pip install flash-attn --no-build-isolation', env={'FLASH_ATTENTION_SKIP_CUDA_BUILD': "TRUE"}, shell=True)
So, there is a way.
1 Like
Thank you very much!!! I will take a look and try it out soon!
