RAG Retriever : Exact vs. Compressed Index?

The compressed index is around 3GB so you don’t need that much RAM actually. The index is the structure used to do retrieval, it doesn’t store the wikipedia texts.

On the other hand the 79GB of data correspond to all the 21M wikipedia texts and their corresponding 768-dim representations. Those data are not loaded in RAM, they’re just memory-mapped from the disk. Memory mapping allows to have fast I/O without filling the RAM.

1 Like