I am considering a PEFT method for summarization. While LoRA is wildly popular, IA3 seems more efficient. I was wondering if there is a benchmark that objectively compares IA3 to LoRA. The results in IA3 paper are rather limited in that they provide results only on a few tasks with smaller models. Please point me to any resource that would be helpful in understanding this better. Thank you
Related topics
| Topic | Replies | Views | Activity | |
|---|---|---|---|---|
| RAG or LoRA for "Talk with Documents" | 4 | 227 | January 16, 2025 | |
| Trainable weights in automodel and comparison with lora | 0 | 217 | April 28, 2023 | |
| Combine LORA with full finetuning | 0 | 386 | September 4, 2023 | |
| Looking for exploratory study / best practices for LoRA adapters config (LLM fine-tuning) | 0 | 362 | April 15, 2024 | |
| Direct Load vs. Base Model + LoRA: How Should You Use It? | 1 | 40 | March 12, 2025 |