CUDA error: CUBLAS_STATUS_NOT_INITIALIZED when calling `cublasCreate(handle)`

[iarbel]

I’m getting the same error, when trying to finetune Llama-7b using PEFT and prefix-tuning. When I change it to LoRA, there is no error