SFTTrainer Loss function

Beginners
1

I have a couple of questions:

  1. how to know the loss function used by default for SFTTrainer for a given model and how to alter it?

  2. For training an LLM the loss function is computed on the whole concatenated prompts, how to alter this and make loss function only compute on the output prompts

1 Like