RAG performance on WebQuestion dataset lower than expected

lkfap · November 18, 2022, 11:15pm

Hi, I recently fine-tuned the Rag model (based on Rag-token-base) in WebQuestion, but the EM score is only 28. The performance in the paper is 45. Do you have any saved models that can match the performance, or does there any tricks during the fine-tuning?

Topic		Replies	Views
Is it mandatory to fine tune a RAG model on custom dataset to generate realted responses for queries, i working on RAG code from the examples , when i use a custom datast=et it doesnot produce intended results for queries Beginners	0	142	September 5, 2023
Although doing RAG does it worth fine tuning the LLM on the documents? - Llama2 Intermediate	1	1190	September 14, 2023
RAG for FEVER Dataset Research	0	343	February 8, 2021
RAG (DPR+seq2seq) pre-trained example Models	0	315	December 12, 2022
How to increase the accuracy of answers from the model that has been fine tuned and uses the RAG and LangChain methods Models	0	788	December 1, 2023

RAG performance on WebQuestion dataset lower than expected

Related Topics