Hey all, I wanted to ask if there is any way to apply LoRA on model and then train model in training loop without
SFTTrainer
I am Using the following way, is it fine?
model_name = "meta-llama/Meta-Llama-3.1-8B-Instruct" # or any other model
model = AutoModelForCausalLM.from_pretrained(
model_name,
# load_in_8bit=True,
# quantization_config=quant_config,
device_map="auto",
max_memory={0: "4GiB", 1: "50GiB", "cpu": "0.0GiB"},
offload_state_dict=True,
low_cpu_mem_usage=True
)
...
model.config.use_cache = False
model.config.pretraining_tp = 1
model.gradient_checkpointing_enable()
model = prepare_model_for_kbit_training(model)
peft_params = LoraConfig(
r=32,
lora_alpha=64,
target_modules=[
"q_proj",
"v_proj",
],
lora_dropout=0.05,
bias="none",
task_type="CAUSAL_LM"
)
model = get_peft_model(model, peft_params)
model.print_trainable_parameters()
optimizer = Adam(model.parameters(), lr=1e-5)
for epoch_idx in [1]:
for batch_idx, batch in tqdm(enumerate(train_data), total=len(train_data)):
optimizer.zero_grad()
outputs = model(**inputs)
...
And then finally using
merge_and_unload()
to merge adapter back to the model
Yes, PEFT is not dependent on the usage of Trainer
or SFTTrainer
. Here is just one example of using LoRA with a custom training loop:
{
"cells": [
{
"cell_type": "code",
"execution_count": 1,
"id": "a9935ae2",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"===================================BUG REPORT===================================\n",
"Welcome to bitsandbytes. For bug reports, please submit your error trace to: https://github.com/TimDettmers/bitsandbytes/issues\n",
"For effortless bug reporting copy-paste your error into this form: https://docs.google.com/forms/d/e/1FAIpQLScPB8emS3Thkp66nvqwmjTEgxp8Y9ufuWTzFyr9kJ5AoI47dQ/viewform?usp=sf_link\n",
"================================================================================\n",
"CUDA SETUP: CUDA runtime path found: /home/sourab/miniconda3/envs/ml/lib/libcudart.so\n",
"CUDA SETUP: Highest compute capability among GPUs detected: 7.5\n",
"CUDA SETUP: Detected CUDA version 117\n",
This file has been truncated. show original
Depending on what you would like to do, custom training loops can get a bit complicated (think: mixed precision, gradient accumulation, etc.) but it’s absolutely possible.
1 Like
Thanks mate. I hope they mention this somewhere the left or Lora page
system
Closed
September 18, 2024, 4:21am
4
This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.