I am training a fine-tune of codellama using PEFT but not sure how to use the task_type parameter of LoraConfig. Should it be CAUSAL_LM or SEQ_2_SEQ_LM or something else? Does it have any affect? The goal of my model is to parse an input for independent clauses in a sentence. For example, it would …

Task_type parameter of LoraConfig

ndvb September 11, 2023, 4:06pm 2

Does task_type matters in LoraConfig, and if so, in what way?

2 Likes