Combine between lora and prompt tunning

🤗Transformers
1

Iam trying to fine tunne LLM using prompt tunning and lora by combining them and start training

1-I freezed both model weights and embedding parameters so i used this :

# freeze the model - train adapters later
for param in model.parameters():
  param.requires_grad = False
  if param.ndim == 1:
    # cast the small parameters (e.g. layernorm) to fp32 for stability
    param.data = param.data.to(torch.float32)

# Freeze original embedding layer
for param in model.config.embeddings.encoder.parameters():
    param.requires_grad = False

model.enable_input_require_grads()

2- I intialized prompt configuration and enjected it with the model:

from peft import PromptTuningConfig, LoraConfig, TaskType, PromptTuningInit, get_peft_model

# Create the PromptTuningConfig object for quiz generation
prompt_config = PromptTuningConfig(
    task_type=TaskType.CAUSAL_LM,
    num_virtual_tokens=10,
    prompt_tuning_init=PromptTuningInit.TEXT,
    prompt_tuning_init_text="", # what your model for: 
    tokenizer_name_or_path=tokenizer
)

model = get_peft_model(model, prompt_config)

3- also intialized the lora configuration:

Lora_config = LoraConfig(
    r=8,
    lora_alpha=16,
    target_modules=["q", "v"],
    lora_dropout=0.05,
    bias="none",
    task_type=TaskType.SEQ_2_SEQ_LM
)

model = get_peft_model(model, Lora_config)

4-then i start training

import transformers

trainer = transformers.Trainer(
    model=model,
    train_dataset="", # your training data
    args=transformers.TrainingArguments(
        per_device_train_batch_size=8,
        gradient_accumulation_steps=8,
        warmup_steps=100,
        max_steps=1000,
        learning_rate=1e-3,
        fp16=True,
        logging_steps=1,
        output_dir='outputs',
    ),
    data_collator=transformers.DataCollatorForLanguageModeling(tokenizer, mlm=False)
)
model.config.use_cache = False  # silence the warnings. Please re-enable for inference!
trainer.train()

at the end all i need to khnow is :

1-Are the additional prompt tokens updated or not , i khnow that the lora wieghts or matrix got updated iam not sure of the prompt tokens

2- since model original weights in lora kept frozen during training , Are the original model embeddings kept frozen too or got changed ?

2

Where you able to verify? and can you also share some results regarding combining prompt tuning + LORA?

Thank you,