Hi,
I would like to deploy SD on the server and accomplish text2img and img2img tasks.
If the image is present, then it would perform img2img, if not, it would do the text2img.
What I would like to do is to have only one model in the memory, so it will not consume
more than is really needed. Is it possible with some easy hack?
Pipe could use a more global manager, with caching to dramatically reduce footprint when not in use after setup, or after a run. But what you should do in the meantime if you’re switching a pipe, is just del pipe and flush torch memory and do a garbage cleanup. Then you can load up the img2img pipe if a init is available, keeping only one in memory.
Another option is to, also do del pipe method, but to employ joblib and dump the pipe to disk, and load it on demand. So you could load up all your pipes, and cache them to disk, and load them just before diffusion, then either dump again (in case of param changes) or just del and load from the same cache before another run. Switch to a img2img cache when init is detected.
The pipe will take same amount of time to set up initially before it’s dumped, but when it loads a pipe, it will take under a second and be ready to feed prompts, etc, and start a run.
A version of what’s described above in case it helps anyone:
You may also find a related GitHub discussion interesting, in particular the code snippet in this comment.
2 Likes