Recently, I need to use the peft library for LoRA fine-tuning, and at the same time, I want to fine-tune certain layers of my base model directly such as lm_head.modules_to_save=["lm_head"] in LoraConfig is a convenient approach, it results in high GPU memory usage. Alternatively, explicitly setting requires_grad=True for lm_head and saving it after training using unload()and model.save_pretrained ("xxx")leads to lower memory consumption. What are the differences between these two approaches in terms of saving the model?
1 Like
In theory, there doesn’t seem to be much difference in the results, but the behavior may differ significantly in practice.save_pretrained() is most likely to be well maintained.
Hello everyone!
Could someone explain why if I add LoRa adapters to a diffusion model using add_adapter method the model layers, which I would like to keep trainable (mentioned in the config parameter modules_to_save), actually are not trainable?
But they are trainable if I use get_peft_model method. Is it a bug or I misunderstand something?
[Screenshot from 2025-01-30 17-15-05]
diffusers 0.32.1
peft 0.14.0
from diffusers import AutoencoderKL
fro…
opened 03:07AM - 12 Apr 25 UTC
closed 11:51AM - 17 Apr 25 UTC
**Summary:**
When using `peft.LoraConfig` with `modules_to_save=["proj_out"]`, … the parameters inside `proj_out` (which are `requires_grad=True` before adapter addition) are unexpectedly frozen (`requires_grad=False`) after calling `add_adapter()`.
This seems to contradict the intended functionality of `modules_to_save`, which should preserve the training status of specified modules.
---
**Environment:**
- `peft==0.15.1` (latest)
- `transformers`, `diffusers` up-to-date
- Model: [`THUDM/CogVideoX-5b-I2V`](https://huggingface.co/THUDM/CogVideoX-5b-I2V)
- Python 3.10
- Torch 2.6.0+cu118
---
**Code to Reproduce:**
```python
from diffusers.models import CogVideoXTransformer3DModel
import torch
from peft import LoraConfig
pretrained_model_name_or_path = "THUDM/CogVideoX-5b-I2V"
transformer = CogVideoXTransformer3DModel.from_pretrained(
pretrained_model_name_or_path,
subfolder="transformer",
torch_dtype=torch.bfloat16,
)
print("before LoRA")
print(transformer.proj_out.weight.requires_grad) # True
print(transformer.proj_out.bias.requires_grad) # True
transformer_lora_config = LoraConfig(
r=4,
lora_alpha=4,
init_lora_weights=True,
target_modules=["to_k", "to_q", "to_v", "to_out.0"],
modules_to_save=["proj_out"]
)
transformer.add_adapter(transformer_lora_config)
print("after LoRA")
print(transformer.proj_out.weight.requires_grad) # ❌ False (unexpected)
print(transformer.proj_out.bias.requires_grad) # ❌ False (unexpected)
```
---
**Expected behavior:**
After calling `add_adapter()`, since `"proj_out"` is explicitly listed in `modules_to_save`, both `.weight` and `.bias` of `transformer.proj_out` should **remain trainable**, i.e., `requires_grad=True`.
---
**Observed behavior:**
Both `.weight` and `.bias` of `proj_out` become `requires_grad=False`, even though `"proj_out"` appears exactly in `named_parameters()` before the adapter is added.
I followed this blog post for saving and loading a PEFT model. I defined my model as
model = AutoModelForSequenceClassification.from_pretrained('roberta-base', return_dict=True)
peft_config = LoraConfig(task_type="SEQ_CLS", inference_mode=False, r=8, lora_alpha=16, lora_dropout=0.1)
model = get_peft_model(model, peft_config)
model.print_trainable_parameters()
I saved it with
save_path = './test_save'
model.save_pretrained(save_path)
and load it with
config = PeftConfig.from_pretrained(save_…