Hi, I was wondering if is there a way to download only part of the data of a dataset. In my specific case, I need to download only X samples from oscar English split (X~100K samples). When I try to invoke the dataset builder it asks for >1TB of space so I think it will download the full set of da…

Hi, let me just complete this: split='train[10:20]' returns a slice of the data, but it still downloads everything. If your dataset is too big, please use streaming mode. You can also slice your dataset in streaming mode, see the documentation here: Stream

Download only a subset of a split

🤗Datasets

morenolq April 4, 2022, 4:41pm 3

Thank you, can I also use streaming mode to reach the same?

Download a fraction of data from HuggingFace Datasets

Topic		Replies	Views
Loading just part of dataset 🤗Datasets	4	4673	February 25, 2025
Loading a fraction of data 🤗Datasets	5	5219	May 12, 2023
How do i load part of the data set Beginners	3	81	May 5, 2025
Downloading a portion of parquet files 🤗Datasets	3	643	May 23, 2024
Streaming in dataset uploads 🤗Datasets	2	50	March 31, 2025

Download only a subset of a split

Related topics