LoRA finetuning for nvidia/NV-Embed-v2

Beginners
1

Problem Summary:
I am trying to apply lora finetuning on the nvidia/NV-Embed-v2 model. The ultimate goal is to use the model for creating embeddings. I am getting an error TypeError: NVEmbedModel.forward() got an unexpected keyword argument 'inputs_embeds'

I checked the files of the model and it does expect the inputs_embed kwarg. I am using FEATURE_EXTRACTION task type for peft config, and using AutoModel to load the model with 8bit quantization. How can I solve for this?

2 Likes
2

same error here. I’m forced to finetune without lora and smaller batch size

1 Like