Hi, I was wondering if is there a way to download only part of the data of a dataset. In my specific case, I need to download only X samples from oscar English split (X~100K samples). When I try to invoke the dataset builder it asks for >1TB of space so I think it will download the full set of da…

Hi, let me just complete this: split='train[10:20]' returns a slice of the data, but it still downloads everything. If your dataset is too big, please use streaming mode. You can also slice your dataset in streaming mode, see the documentation here: Stream

Download only a subset of a split

🤗Datasets

morenolq April 4, 2022, 4:41pm 3

Thank you, can I also use streaming mode to reach the same?

Download a fraction of data from HuggingFace Datasets

Topic		Replies	Views
Loading a fraction of data 🤗Datasets	5	5278	May 12, 2023
Loading just part of dataset 🤗Datasets	4	4797	February 25, 2025
Download a fraction of data from HuggingFace Datasets 🤗Datasets	4	285	November 20, 2024
Downloading a portion of parquet files 🤗Datasets	3	667	May 23, 2024
How do i load part of the data set Beginners	3	91	May 5, 2025

Download only a subset of a split

Related topics