Hello, I have recently fine-tuned the Zephyr 7B model with a dataset consisting of approximately 2000 lines of legal articles with input-output formats. The fine-tuning results show that the model can answer according to context and task, but the accuracy is not satisfactory.
I tried to add the RAG method with embeddings using LangChain and the bge-large model, but this actually decreased the accuracy of the answers. I have tried changing the embedding model as well as changing the chunk parameters. But It seems that the embedding model is unable to find relevant information from the input documents.
Does anyone have any suggestions or solutions for this issue? I would also like to know if methods like gating or weighting can be applied to HuggingFaceEmbeddings in LangChain. Thank you for your help