Program not working on GPU but works on CPU

Intermediate
9

The simple code

from diffusers import DiffusionPipeline
import torch

pipe = DiffusionPipeline.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0",
    torch_dtype=torch.float32
).to("cuda")

prompt = "A clear sunny landscape with mountains and a river"
image = pipe(prompt=prompt).images[0]
image.save("test_image.png")

returns the error

(venv) D:\Ganu\AIImage\project\Train-10Images-chatgptParameters\runs\1sstrun-23thApril2025\generation\1stGo>python John-Training-15thMay2025.py
[2025-05-15 14:53:54,560] [INFO] [real_accelerator.py:133:get_accelerator] Setting ds_accelerator to cuda (auto detect)
D:\Ganu\AIImage\venv\lib\site-packages\deepspeed-0.10.0+f5c834a6-py3.10.egg\deepspeed\runtime\zero\linear.py:49: FutureWarning: `torch.cuda.amp.custom_fwd(args...)` is deprecated. Please use `torch.amp.custom_fwd(args..., device_type='cuda')` instead.
  def forward(ctx, input, weight, bias=None):
D:\Ganu\AIImage\venv\lib\site-packages\deepspeed-0.10.0+f5c834a6-py3.10.egg\deepspeed\runtime\zero\linear.py:67: FutureWarning: `torch.cuda.amp.custom_bwd(args...)` is deprecated. Please use `torch.amp.custom_bwd(args..., device_type='cuda')` instead.
  def backward(ctx, grad_output):
Loading pipeline components...: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:02<00:00,  2.48it/s]
Traceback (most recent call last):
  File "D:\Ganu\AIImage\project\Train-10Images-chatgptParameters\runs\1sstrun-23thApril2025\generation\1stGo\John-Training-15thMay2025.py", line 7, in <module>
    ).to("cuda")
  File "D:\Ganu\AIImage\venv\lib\site-packages\diffusers\pipelines\pipeline_utils.py", line 733, in to
    module.to(torch_device, torch_dtype)
  File "D:\Ganu\AIImage\venv\lib\site-packages\transformers\modeling_utils.py", line 1900, in to
    return super().to(*args, **kwargs)
  File "D:\Ganu\AIImage\venv\lib\site-packages\torch\nn\modules\module.py", line 1174, in to
    return self._apply(convert)
  File "D:\Ganu\AIImage\venv\lib\site-packages\torch\nn\modules\module.py", line 780, in _apply
    module._apply(fn)
  File "D:\Ganu\AIImage\venv\lib\site-packages\torch\nn\modules\module.py", line 780, in _apply
    module._apply(fn)
  File "D:\Ganu\AIImage\venv\lib\site-packages\torch\nn\modules\module.py", line 780, in _apply
    module._apply(fn)
  File "D:\Ganu\AIImage\venv\lib\site-packages\torch\nn\modules\module.py", line 805, in _apply
    param_applied = fn(param)
  File "D:\Ganu\AIImage\venv\lib\site-packages\torch\nn\modules\module.py", line 1160, in convert
    return t.to(
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 242.00 MiB. GPU 0 has a total capacity of 4.00 GiB of which 0 bytes is free. Of the allocated memory 10.41 GiB is allocated by PyTorch, and 262.02 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)

and chatgpt says

You're running out of VRAM on your GPU (4 GB total, 0 bytes free), and you're trying to load Stable Diffusion XL (SDXL), which requires at least 12 GB of GPU memory, ideally more. The error is expected — SDXL is far too heavy for a 4 GB GPU.

P.s:

  1. The cpu version still works.
  2. And I remember the gpu version working earlier
  3. Do you want me to change the ( CUDA Toolkit version )
1 Like