Darshan Hiranandani : What methods are available for updating a saved dataset with new rows?

🤗Datasets
1

Hi Team,

I’m Darshan Hiranandani, looking to update a saved dataset by adding new rows to it. Could anyone share their suggestions or methods for accomplishing this? Specifically, I’m interested in approaches or tools that can efficiently handle this task, especially when dealing with large datasets.

Any insights, code snippets, or recommended libraries would be greatly appreciated.

Thanks for your help!

Darshan Hiranandani

2

Hi. You might have some insights here: Repository limitations and recommendations

See also Incremental dataset (e.g. `.push_to_hub(..., append=True)`) · Issue #6290 · huggingface/datasets · GitHub → not implemented yet, but definitely something we want to have at some point. Don’t hesitate to +1 or comment there.