Getting an error in AutoTrain

John6666 · January 22, 2025, 4:39am

This is a bitsandbytes error, but the solution may be to reinstall PyTorch. Installing PyTorch is the first hurdle in generative AI…

github.com/bitsandbytes-foundation/bitsandbytes

An error occurred: CUDA is required but not available for bitsandbytes.

opened 05:11PM - 09 Oct 24 UTC

GaoDalie

### System Info please I have tried many ways but I couldn't address the issues…, could anyone please give me a hint or help me to solve this bug because I couldn't figure it where the problem coming from note: I have installed Cuda in my env, but I am still getting an error here is the error : An error occurred: CUDA is required but not available for bitsandbytes. Please consider installing the multi-platform enabled version of bitsandbytes, which is currently a work in progress. Please check currently supported platforms and installation instructions at https://huggingface.co/docs/bitsandbytes/main/en/installation#multi-backend thank you so much ### Reproduction import torch from transformers import AutoModelForVision2Seq, AutoProcessor, BitsAndBytesConfig # Hugging Face model id try: model_id = "Qwen/Qwen2-VL-7B-Instruct" # BitsAndBytesConfig int-4 config bnb_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_use_double_quant=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.bfloat16 ) # Load model and tokenizer model = AutoModelForVision2Seq.from_pretrained( model_id, device_map="auto", torch_dtype=torch.bfloat16, quantization_config=bnb_config ) except Exception as e: print(f"An error occurred: {e}") ### Expected behavior please I have tried many ways but I couldn't address the issues, could anyone please give me a hint or help me to solve this bug because I couldn't figure it where the problem coming from note: I have installed Cuda in my env, but I am still getting an error here is the error : An error occurred: CUDA is required but not available for bitsandbytes. Please consider installing the multi-platform enabled version of bitsandbytes, which is currently a work in progress. Please check currently supported platforms and installation instructions at https://huggingface.co/docs/bitsandbytes/main/en/installation#multi-backend thank you so much

Topic		Replies	Views
Meta Llama - CUDA Error Beginners	1	161	January 22, 2025
I need help with AutoTrain Beginners	1	20	February 11, 2025
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:2 and cuda:0! (when checking argument for argument index in method wrapper_CUDA__index_select) DeepSpeed	5	3357	August 26, 2024
Autotrain not working 🤗AutoTrain	0	318	April 10, 2024
Getting runtime error when using AutoTrain 🤗AutoTrain	3	1034	December 22, 2023

Getting an error in AutoTrain

Related topics