Hello All, I am trying to load meta/opt6.7B in Kaggle with dual T4 GPU. The code is simple and I could run that in colab. But I can’t make it work in Kaggle. What am I missing? Here is the code - !pip install bitsandbytes datasets accelerate loralib !pip install transformers peft import os # os…

Unable to load 8bit model in Kaggle with dual GPU

rcshubhadeep March 26, 2023, 5:05am 2

Ok this was super easy actually. I needed to upgrade the packages. Kaggle comes with pre-installed versions and they were old. So that is that. I have solved it

1 Like

Topic		Replies	Views
Issues with loading models in 8bit in Colab Beginners	0	335	December 13, 2023
Unable to inference in 8bit mode: 'NoneType' object has no attribute 'device' 🤗Transformers	4	2291	December 14, 2023
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:1 and cuda:0! 🤗Transformers	28	114861	November 17, 2024
"normal_kernel_cpu" not implemented for 'Char' when trying to import 8-bit model 🤗Transformers	6	1908	February 23, 2025
Bitsandbytes and CUBLAS_STATUS_NOT_INITIALIZED error Beginners	21	49	August 29, 2025

Unable to load 8bit model in Kaggle with dual GPU

Related topics