Hello everyone,
I have a datasets.Dataset created with the following features and size.
Dataset(features: {'embeddings': Sequence(feature=Value(dtype='float64', id=None), length=-1, id=None), 'pid': Value(dtype='int32', id=None)}, num_rows: 8841823)
I added a FAISS index over ‘embeddings’ column in the GPU.
docs.load_faiss_index('embeddings', DATAPATH_INDEXED_ENCODED_PASSAGES, device=torch.cuda.current_device())
RuntimeError: Error in void faiss::gpu::allocMemorySpaceV(faiss::gpu::MemorySpace, void**, size_t) at gpu/utils/MemorySpace.cpp:26: Error: 'err == cudaSuccess' failed: failed to cudaMalloc 27162080256 bytes (error 2 out of memory)
I am working on a NVIDIA Tesla K80 with 1 GPU having 11.4GB memory. Well, it seems that with this hardward is impossible to index it on the GPU. Running the index on the CPU leads to infeasible execution time on the next step since I would like to get the top 1000 nearest examples for 6900 queries.
Any workarounds? I’m not sure if I’ll have the resources, but if I get access to several GPUs like the one that I’m working with, I just need to specify multiple GPU IDs when loading the faiss index, right?