Hallucinations after fine tuning data augmentated dataset even with low loss

Beginners
1

Hi !

I tried to fine tuning using this dataset: https://huggingface.co/datasets/celsowm/enunciados_pge_rj_orpo created using some techniques of data augumentation using as source this simples page: https://pge.rj.gov.br/entendimentos/enunciados

I tried llama3.2 1B using SFT + lora and even ORPO, all with low loss.

But when I prompt something present in the dataset like: “Qual é o conteúdo do Enunciado 47 da PGE-RJ?.” (What is the content of Statement 47 of the PGE-RJ?)

The response is a Hallucination from another Enunciado (Statement);

Unfortunately my 3060 ti 12gb is not enough to full fine tuning so I did not try yet.

Any hints???

Thanks in advance !

1 Like