Hi, I was wondering if is there a way to download only part of the data of a dataset. In my specific case, I need to download only X samples from oscar English split (X~100K samples). When I try to invoke the dataset builder it asks for >1TB of space so I think it will download the full set of da…

Hi, let me just complete this: split='train[10:20]' returns a slice of the data, but it still downloads everything. If your dataset is too big, please use streaming mode. You can also slice your dataset in streaming mode, see the documentation here: Stream

Download only a subset of a split

🤗Datasets

morenolq April 4, 2022, 4:41pm 3

Thank you, can I also use streaming mode to reach the same?

Download a fraction of data from HuggingFace Datasets

Topic		Replies	Views
Loading just part of dataset 🤗Datasets	4	3876	February 25, 2025
Loading a fraction of data 🤗Datasets	5	4727	May 12, 2023
Downloading a portion of parquet files 🤗Datasets	3	540	May 23, 2024
Download only 1 of many parquet file 🤗Datasets	2	143	March 19, 2025
Can I download only the first half of a large dataset? 🤗Datasets	1	306	May 29, 2024

Download only a subset of a split

Related topics