Using datasets 1.8.0. Normal? Situations If I use load_dataset() to load data, it generates cache files. If you then apply .map() on that dataset, corresponding cache files are generated as expected. Following is a simple code to reproduce the results. from datasets import load_dataset, Dataset …

Caching a dataset with map() when loaded with from_dict()

eli100 March 22, 2023, 6:27pm 4

Also stumbled on this bug. Any plans to fix it?

Topic		Replies	Views
Dataset map() creates lot of cache files 🤗Datasets	6	6778	March 26, 2024
The datasets.map function does not load cached dataset Beginners	7	2339	November 21, 2023
Load dataset from a specific cache file 🤗Datasets	3	1376	February 26, 2024
Increase on disk space when using map() in Accelerate environment 🤗Datasets	2	1195	August 18, 2022
Pipeline with custom dataset tokenizer: when to save/load manually 🤗Datasets	18	5751	September 18, 2020