How to load a pretrained PEFT parameters into an trl model?

🤗Transformers
1

I have pre-trained a LoRA model using the PEFT method, and now I have a similar task where I need to use the PPO training process. I would like to initialize the LoRA parameters in the AutoModelForCausalLMWithValueHead model with the pre-trained LoRA parameters. How should I do this?

1 Like