Hugging Face Forums
OOM issue with large dataset streaming
🤗Datasets
John6666
March 6, 2025, 4:06pm
2
Is the data set file itself too large…?
If it seems like a bug, let’s call lhonestq.
show post in topic
Related topics
Topic
Replies
Views
Activity
How to load a large hf dataset efficiently?
🤗Datasets
5
2197
January 22, 2024
Streaming in dataset uploads
🤗Datasets
2
40
March 31, 2025
"too many open files" despite streaming with IterableDataset
🤗Datasets
2
25
January 27, 2025
Using large dataset with accelerate
🤗Accelerate
0
32
March 6, 2025
Recommended max size of dataset?
🤗Datasets
5
66
March 11, 2025