Fine tune with SFTTrainer

That looks like an issue with data preparation. Are you using the tokenizer to prepare data for the model?