Upload Sentence Transformer embeddings

Beginners
4

Sorry, I missed your post!

Any limit on storage size?

We don’t really have a limit! You might want to shard your files though, more on that here: Is there a size limit for dataset hosting - #4 by julien-c

For the other questions:

  1. As far as I’m aware, you can just set df["embeddings"] = embeddings or something like that, and it should be fine
  2. If you’ve stored each embedding like above, when you retrieve the particular subset of the dataframe for your operation you can just call df["embeddings"] again.

Hope I understood your question – let me know if this didn’t help!