I’ve noticed that it consistently takes ~2 minutes for a paused Space to start booting. That is, it takes about 2 minutes for the app server code to start running, non-inclusive of the actual app server startup time. This initial delay seems to be independent of the actual content of the Space.
Is there any way of bringing this time down? A 2 minute cold boot is a non-starter for our use case. Keeping the Spaces running all the time would be expensive, and we’d rather avoid doing that.
For reference, I’m using the built-in Gradio-backed Spaces, and experiencing this same latency across a range of underlying hardware and Spaces.
Unfortunately, avoiding a cold start here is no possible. However, one way to accelerate your Space boot is by using persistent storage to cache all large models and data. As result, the cold start will benefit from the cached Docker image, removing the need to re-download large files. The pre-cached image will boot up very quickly, and the /data mounted at runtime, you will minimize latency during model loading to memory.