Build timeout when starting space

Beginners
1

Hi, I duplicated threestudio - a Hugging Face Space by bennyguo. I’ve tried to start it a couple of times and it always ends with…

Build error

Build timeout

…with no obvious errors in the logs. Is there some time limit to building the space that I can change in settings? I set the sleep settings to 10 hours so it’s not that.

2

hi @neilmendoza :wave:
Indeed, I’m trying now the duplication with a t4 GPU, and it’s stuck building nerfacc python wheel.
We do have the undocumented parameter for the README.md yaml startup_duration_timeout: 1h, However, you can try replacing the build with a pre-compile wheel

Here

Line 55 on the Dockerfile

RUN pip install nerfacc==0.5.2 -f https://nerfacc-bucket.s3.us-west-2.amazonaws.com/whl/torch-2.0.0_cu118.html

3

never mind, since it stills need to build tinycudann so better try with startup_duration_timeout: 1h

4

Hi @radames!! :raising_hand_man:

Running it on an A10G as seems like you need 20GB of VRAM to run some of the bigger models. Think it will actually take longer than an hour to build so stuck startup_duration_timeout: 2h in README.md as seems like where the settings go (complete noob here).

5

hi @neilmendoza , not it successfully build on a T4 16gb VRAM if you’re using Stable Diffusion, if you need DeepFloyd then you need >20VRAM

# if you have agreed the license of DeepFloyd IF and have >20GB VRAM
# please try this configuration for higher quality
python launch.py --config configs/dreamfusion-if.yaml --train --gpu 0 system.prompt_processor.prompt="a zoomed out DSLR photo of a baby bunny sitting on top of a stack of pancakes"
# otherwise you could try with the Stable Diffusion model, which fits in 6GB VRAM
python launch.py --config configs/dreamfusion-sd.yaml --train --gpu 0 system.prompt_processor.prompt="a zoomed out DSLR photo of a baby bunny sitting on top of a stack of pancakes"

Need the following patches
Dockerfile#L55
Dockerfile#L65

6

Thanks! I set startup_duration_timeout: 4h in the README.md and also used your patches but still seems to time out when building :thinking:

7

hi @neilmendoza ,
just tried again on a A10, 100s to the first pass, building tiny-nn.
Please try factory reboot.

here’s the Docker build log

..
DONE 102.2s

--> ADD https://api.github.com/repos/threestudio-project/threestudio/commits?per_page=1 latest_commit
DONE 0.1s

--> RUN git clone https://github.com/threestudio-project/threestudio.git /home/dreamer/threestudio
Cloning into '/home/dreamer/threestudio'...
DONE 1.0s

--> WORKDIR /home/dreamer/threestudio
DONE 0.0s

--> RUN git checkout 27d69d9845016c8b8aa0bac92ab6d4fea8d1e1b8
Note: switching to '27d69d9845016c8b8aa0bac92ab6d4fea8d1e1b8'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 27d69d9 update zero123 simple config
DONE 0.2s

--> Pushing image
DONE 33.6s

--> Exporting cache
DONE 8.8s