Dataset repo requires arbitrary Python code execution

severo · November 30, 2023, 12:05pm

Indeed. We have an issue to handle that case, feel free to chime in, or +1.

Multi-image loading in Imagefolder dataset

opened 04:01PM - 16 Apr 23 UTC

enhancement

### Feature request Extend the `imagefolder` dataloading script to support load…ing multiple images per dataset entry. This only really makes sense if a metadata file is present. Currently you can use the following format (example `metadata.jsonl`: ``` {'file_name': 'path_to_image.png', 'metadata': ...} ... ``` which will return a batch with key `image` and any other metadata. I would propose extending `file_name` to also accept a list of files, which would return a batch with key `images` and any other metadata. ### Motivation This is useful for example in segmentation tasks in computer vision models, or in text-to-image models that also accept conditioning signals such as another image, feature map, or similar. Currently if I want to do this, I would need to write a custom dataset, rather than just use `imagefolder`. ### Your contribution Would be open to doing a PR, but also happy for someone else to take it as I am not familiar with the datasets library.

Topic		Replies	Views
Dataset Viewer for dataset with downloadable data 🤗Datasets	3	31	March 6, 2025
Dataset With Script Not Supported Error 🤗Datasets	1	94	March 29, 2025
Enabling dataset viewer by coexistence of loading script and parquet files 🤗Datasets	5	319	March 18, 2024
My dataset viewer is not loading 🤗Datasets	2	39	January 17, 2025
Dataset preview not showing for uploaded DatasetDict 🤗Datasets	6	2139	December 7, 2021

Dataset repo requires arbitrary Python code execution

Related topics