to add to what @shamanez said, i also recently learnt that you can add faiss indices directly in the datasets
library: Adding a FAISS or Elastic Search index to a Dataset — datasets 1.6.0 documentation
to add to what @shamanez said, i also recently learnt that you can add faiss indices directly in the datasets
library: Adding a FAISS or Elastic Search index to a Dataset — datasets 1.6.0 documentation