I am a college student racking my brain over my thesis project. My task is to fine-tune Llama3.1-8b using 50,000 QA pairs.
The QA data comes from the following source:
https://huggingface.co/datasets/aigrant/medical_health
I am extracting only the instruction, input, and output fields and organizing them into a .json file in the following format:
[
{
“instruction”: “了解過敏性鼻炎”,
“input”: “過敏性鼻炎的趨勢和症狀”,
“output”: “過敏性鼻炎患者逐年增加，孩童約有一半會有鼻子問題，成人約20%有症狀。症狀包括鼻癢、打噴嚏、流鼻水、鼻塞等。”
},…
We plan to use this on Ollama. I am fine-tuning with Unsloth and attempting to run it on either Colab or my local machine. However, over the past three months, no matter how I fine-tune it—despite following various online tutorials—the .gguf file I generate not only fails to work on Ollama (create fails) but also doesn’t work on other software. The responses turn into gibberish or something resembling multiple-choice questions.
If anyone can guide me or suggest an alternative solution, I would greatly appreciate it.
Currently, I am using an RTX 3050 Ti Laptop GPU, and our lab has an RTX 3090 available for use.