Loss Issues on Finetuning

[hayalolsam]

Hello,

Sometimes while finetuning an LLM, the train and validation loss decreases smoothly but the model does not learn the behavior properly. What could be the reasons for this? Could it be related to the decoding strategy? Or Dataset?

Thank you.