I have use PEFT lora technique to fine tune a mistral model i used bit and byte for my quantization so wanted to where its saving ,mu quantize model and how i can use quantize model with my adapter layer. if any reference please share article
Related topics
Topic | Replies | Views | Activity | |
---|---|---|---|---|
Finetuned LLM model conversion to GGUF - performance drop | 4 | 1848 | July 31, 2024 | |
How to load a model fine-tuned with QLoRA | 2 | 6666 | July 29, 2024 | |
Peft model from pretrained load in 8/4 bit | 6 | 17565 | October 12, 2023 | |
Quantizing a model on M1 Mac for qlora | 0 | 1672 | March 14, 2024 | |
Peft following bits and bytes seems to have no effect on LLM | 0 | 496 | January 31, 2024 |