I know what is AutoModelForCausalLM
. The thing I’m asking is that in the peft
LoRA Fine tuning tutorial, the autors have used AutoModelForCausalLMWithValueHead
while you pick any code or notebook on Fine-tuning of any LLM with PEFT
style, you’ll find AutoModelForCausalLM
being used.
I went to lean on the official documentation of AutoModelForCausalLMWithValueHead
and found:
An autoregressive model with a value head in addition to the language model head
What I want to ask is that How, where and more importantly, WHY this extra ValueHead
is used