I wonder what’s the proper way to “save” a transformed tensor dataset so that we can reload it next time without going through the transformation again?
doc_emb = model.encode(docs, convert_to_tensor=True)
I’ve tried to save it as a pickle file like I did on my local PC but it seems that this approach does not work on the huggingface space?
I wonder what’s the proper way to save and reload it?