Hugging Face Forums
Can I load a model fine-tuned with LoRA 4-bit quantization as an 8-bit model?
🤗Hub
supercoolaj
November 27, 2023, 7:02am
1
Or do I have to load it as a 4-bit each time? Thanks.
Related Topics
Topic
Replies
Views
Activity
Fine tuning LoRa merge
Beginners
0
271
January 10, 2024
Loading quantised weights does not work
Beginners
0
115
April 12, 2024
Error. Model cannot be quantized if a LoRA adapter has been applied to it via merge_and_unload()
Beginners
0
177
May 12, 2024
Low bf16 performance on TPU, int4 vs int8 quantizatoin
🤗Accelerate
0
177
June 1, 2024
Loading an LoRA adapter trained on quantized model on a non-quantized model
Intermediate
0
1056
November 7, 2023