I am trying to implement diffusers as workers using Celery. I am also aiming to make it work in Single GPU, Multi GPU and TPU environments with no code adjustments and only configuration.
As with Celery, if the code is assumed to work in
tool_specific_command mycode.py format instead of
python3 mycode.py, it will be a big problem when trying to use two different frameworks at the same time. Fortunately, Celery has a way to make it work the same way as if you had used commands, only with coding instead of commands.
Does such a path exist for accelerate?