Hi, I’m new to the Huggingface’s Datasets and I’m trying to train controlnet for stablediffusion on a custom dataset with approximately 300k images, the size of these images is (768, 768). Now, I stuck in following lines of code: dataset = load_dataset("imagefolder", data_dir="path/to/the/dataset"…

Extremely slow data loading of imagefolder

kopyl December 19, 2023, 4:31am 4

i’m basically loading it to upload to HF… So it still need to download it, right?

Topic		Replies	Views
Creating a HF Dataset from lakeFS with S3 storage takes too much time! 🤗Datasets	7	87	June 23, 2025
Best practice loading images files 🤗Datasets	3	1676	March 27, 2024
Allow streaming of large datasets with image/audio 🤗Datasets	18	4044	May 30, 2022
5M small images (~100Gb) 🤗Datasets	2	526	July 5, 2022
How to load large-scale text-image pair dataset 🤗Datasets	4	1054	February 7, 2025