Hi !
I tried to fine tuning using this dataset: https://huggingface.co/datasets/celsowm/enunciados_pge_rj_orpo created using some techniques of data augumentation using as source this simples page: https://pge.rj.gov.br/entendimentos/enunciados
I tried llama3.2 1B using SFT + lora and even ORPO, all with low loss.
But when I prompt something present in the dataset like: “Qual é o conteúdo do Enunciado 47 da PGE-RJ?.” (What is the content of Statement 47 of the PGE-RJ?)
The response is a Hallucination from another Enunciado (Statement);
Unfortunately my 3060 ti 12gb is not enough to full fine tuning so I did not try yet.
Any hints???
Thanks in advance !