Task_type parameter of LoraConfig

raptorkwok · October 17, 2023, 8:54am

In the source code, task_type is not even there:

huggingface/peft/blob/15a013af5ff5660b9377af24d3eee358213d72d4/src/peft/tuners/lora/config.py#L24


      
          # limitations under the License.
          
          from dataclasses import dataclass, field
          from typing import List, Optional, Union
          
          from peft.config import PeftConfig
          from peft.utils import PeftType
          
          
          @dataclass
          class LoraConfig(PeftConfig):
              """
              This is the configuration class to store the configuration of a [`LoraModel`].
          
              Args:
                  r (`int`): Lora attention dimension.
                  target_modules (`Union[List[str],str]`): The names of the modules to apply Lora to.
                  lora_alpha (`int`): The alpha parameter for Lora scaling.
                  lora_dropout (`float`): The dropout probability for Lora layers.
                  fan_in_fan_out (`bool`): Set this to True if the layer to replace stores weight like (fan_in, fan_out).
                      For example, gpt-2 uses `Conv1D` which stores weights like (fan_in, fan_out) and hence this should be set

Topic		Replies	Views
LoraConfig task_type 🤗Transformers	0	613	September 11, 2023
LoRa Task Type what is the difference between Seq2Seq and CausalLM 🤗Transformers	0	1027	May 5, 2023
Fine Tune with/without LORA 🤗Transformers	1	253	October 7, 2024
`get_peft_model` or `model.add_adapter` Beginners	2	1240	February 17, 2025
Combine LORA with full finetuning Intermediate	0	397	September 4, 2023

Task_type parameter of LoraConfig

Related topics