Hello everyone.
I need help fine-tuning Llama3 to analyse exception messages from log files generated by a Windows application.
I created a dataset in huggingface with a lot of possible inputs, which are the exception messages, and the outputs, which are divided into 7 categories:
- Coding issue
- Network issue
- Database issue
- Infrastructure issue
- Memory issue
- Service issue
- Irrelevant
However, the dataset is very small and when I’m testing the model I’m noticing either wrong answers or a lot of hallucinations.
For example: If I ask it to tell me the type of problem a certain ExcpMessage represents it starts answering with the “instruction” and “input” fields over and over, instead of only the “output”
I’m also using Unsloth and the Colab code to fine-tune Llama3.
The dataset: https://huggingface.co/datasets/agoncalves/log_excp_messages . There are only 146 rows as I can not find anymore examples to feed the model.
Important to note that the only thing I’ve changed in the Colab code was removing the “instruction” variable, adding temperature=0 and changing the dataset from alpaca to mine.
Should I change something else? Or perhaps changing the dataset? I’m still a beginner and there are a lot of things I don’t understand.
Any tips are welcome.
Thank you