Hey all, I wanted to ask if there is any way to apply LoRA on model and then train model in training loop without SFTTrainer I am Using the following way, is it fine? model_name = "meta-llama/Meta-Llama-3.1-8B-Instruct" # or any other model model = AutoModelForCausalLM.from_pretrained( model…

Yes, PEFT is not dependent on the usage of Trainer or SFTTrainer. Here is just one example of using LoRA with a custom training loop: huggingface/peft/blob/main/examples/sequence_classification/LoRA.ipynb { "cells": [ { "cell_type": "code", "execution_count": 1, "id": "a…

Training loop for LoRA

🤗Transformers

khatrimann September 17, 2024, 4:21pm 3

Thanks mate. I hope they mention this somewhere the left or Lora page

Topic		Replies	Views
"You cannot perform fine-tuning on purely quantized models." error in LoRA model training? 🤗Transformers	3	2714	August 16, 2024
`get_peft_model` or `model.add_adapter` Beginners	2	1241	February 17, 2025
Loading Lora models after trainning Beginners	1	3375	June 24, 2024
Fine Tune with/without LORA 🤗Transformers	1	253	October 7, 2024
Missing trainable parameters in a loaded LoRA model 🤗Transformers	1	1317	July 6, 2023

Training loop for LoRA

Related topics