Load_dataset without saving cache files

🤗Datasets
1

I want to load the dataset without saving cache file or delete the cache file after use.
so i use cleanup_cache_files() function after use dataset file likes:

from datasets import load_dataset
dataset = load_dataset('cc100',lang='ar',split='train')
dataset.cleanup_cache_files()

but it didn’t delete cache file in .cache/huggingface/datasets
How can I avoid storing cache files or delete them automatically after use?

2

cleanup_cache_files removes the arrow cache files of the Dataset

However the downloaded files used to prepare the arrow cache files are still in .cache/huggingface/datasets/downloads that you can delete if you want

1 Like