Hello, how do I implement QA-LoRA with Pytorch? There is code in the paper of QA-LoRA, but I have no idea, how to implement the code. I want to fine-tune models like BERT or Llama-2 with QA-LoRA. Thanks in advance.

Implentation of QA-LoRA

solomonpm December 26, 2023, 4:27pm 2

Hi team,
I am having similar requirement.

Like to use QA-LoRA to fine tune models like owl-ViTv2 and Grounding Dino.

Sample reference code will be useful.

with thanks

1 Like

Topic		Replies	Views
AWQ model LoRA fine-tuning Beginners	1	480	April 8, 2024
Using SQuAD & LoRa to Fine tuning gpt2 for QA task Beginners	0	407	June 14, 2024
How to fine tune a LORA fine tuned model 🤗Transformers	0	304	July 14, 2023
Fine tuning for Llama2 based model with LoftQ quantization 🤗Transformers	7	2383	January 24, 2024
Fine tuning using llm Qlora Beginners	0	915	March 20, 2024