Hi,
Thank you for the new version of the rag model; it’s really nice. I’ve tried to install the required packages and set everything up using the code snippet below
from transformers import AutoTokenizer, RagRetriever, RagModel
import torch
tokenizer = AutoTokenizer.from_pretrained(“facebook/rag-token-base”)
retriever = RagRetriever.from_pretrained(“facebook/rag-token-base”, index_name=“exact”)
model = RagModel.from_pretrained(“facebook/rag-token-base”, retriever=retriever)
inputs = tokenizer(“what is physics?”, return_tensors=“pt”)
outputs = model(input_ids=inputs[“input_ids”])
print(outputs)
It has downloaded all the required packages, which total 80 GB in size. Afterward, it initiated the indexing process for ‘wiki_dpr.’ which is total of 75 GB in size. My question is whether this indexing step will occur every time we run the code, and also, the ‘wiki_dpr’ data is stored in my local cache folder. How can I load this data into the retriever? If we want to load data from datasets how we can load small dataset, noticed that above code is going for full indexing every time we run and it is taking 12+ hr in windows machine could you please provide guidance on how to set up the RAG model?
Thank you in advance.