We check available disk space using this logic (with directory being equal to Path(load_dataset_builder("file_location").cache_dir).parent).
I think I have enough space on my current directory but not enough space in my home directory.
Then pass this “current directory” as a cache_dir to load_dataset. Otherwise, generating the dataset defaults to ~/.cache/huggingface/datasets.