Hello , I new into this space , few days ago i tried unsloth to fine tine , i followed a yt video and used the unsloths Llama 3.1 8b model , and after training or fine tuning (pardon for using the wrong terms) when i used the (fine tuned)model on the colab notebook the responses were amazing but when i downloaded it locally the responses were horrible and sometimes the model won’t give any response (was using it on ollama ) but after that i ran that same local gguf model on lm studio it is giving perfect response but seems overfitted . So I am thinking to do the fine tuning again but this time locally but my 3050 has 4gb VRAM only and my Laptop has 16gb ram I don’t care if the process will be slow but want to know that can i do the fine tuning (QLoRA) on this specs
1 Like
I don’t know if it’s possible to go with that VRAM size, but the following may be helpful for reducing VRAM and RAM in training.