.to("cuda"),The model cannot be moved to the GPU

Spaces
1

this is my code,my envs :pytorch == 2.7.1,cuda=12.8 python==3.11,ubuntu20.04, 4090-24G
After testing, I found that the model couldn’t be moved to the GPU. The.to(" cuda ") operation couldn’t be carried out and was stuck at this step, but no error was reported

import torch
from diffusers import FluxFillPipeline
from diffusers.utils import load_image

image = load_image("https://huggingface.co/datasets/diffusers/diffusers-images-docs/resolve/main/cup.png")
mask = load_image("https://huggingface.co/datasets/diffusers/diffusers-images-docs/resolve/main/cup_mask.png")
#device = "cuda" if torch.cuda.is_available() else "cpu"

print("Loading model...")
pipe = FluxFillPipeline.from_pretrained("black-forest-labs/FLUX.1-Fill-dev", torch_dtype=torch.bfloat16)
print("Moving to GPU")
pipe = pipe.to("cuda")

print("Has moving to GPU ")

print("Model loaded successfully!")


print("Starting inference...")
image = pipe(
    prompt="a white paper cup",
    image=image,
    mask_image=mask,
    height=1632,
    width=1232,
    guidance_scale=30,
    num_inference_steps=50,
    max_sequence_length=512,
    generator=torch.Generator("cpu").manual_seed(0)
).images[0]

print("Inference completed.")
image.save(f"flux-fill-dev.png")
1 Like
2

4090-24G

It has three times the VRAM of my GPU, but this is the reason. FLUX is approximately 36 GB in size. Let’s quantize it.