I have been trying to finetune Llama 3 8b Instruct on new legal laws such as Bharatiya Nyaya Sanhita replacing the indian penal code but the responses were not great as the model cited wrong section from the laws. My dataset size 1600 as these are the total number section of it.so what would you suggest to do
1 Like
I would say that with a dataset size of 1600, it might not be enough data to fine tune the model. If possible, you might want to try and increase the size of the dataset. If that’s not possible, I would just use RAG instead of fine tuning the whole model.
1 Like