I am trying to use ISTA-DASLab/Meta-Llama-3.1-70B-AQLM-PV-2Bit-1x16 in the zero space KwabsHug/TestCompressedModelzero. When you press generate it starts but always returns OSError: CUDA_HOME environment variable is not set. Please set it to your CUDA install root. Does anyone know how resolve thi…

Okay, it works! But no matter how you look at it, I just forced it to work.:sweat_smile: import spaces import gradio as gr from transformers import AutoTokenizer, AutoModelForCausalLM import torch import subprocess import os def install_cuda_toolkit(): # CUDA_TOOLKIT_URL = "https://developer.…

Error running model in zerogpu

Spaces

John6666 October 3, 2024, 12:27am 5

Well, I don’t think the fundamental problem has been solved, but I guess it’s better to have it work than not to have it work!
It’s a bug, this…

Topic		Replies	Views
CUDA_HOME on zerogpu Beginners	3	1102	November 13, 2024
Torch.cuda.is_available() is False on ZeroGPU Space Spaces	3	34	April 22, 2025
Error while initializing ZeroGPU Models	13	227	August 10, 2025
Cuda Error with Zero Space Beginners	2	121	December 11, 2024
ZeroGPU space : No CUDA GPUs are available Spaces	4	157	May 14, 2025

Error running model in zerogpu

Related topics