I want to download dataset, but using the huggingface cli module, I can’t seem to avoid API. I want to avoid git commands, as they tend to cause OOM when used in a docker container with limited resources, when downloading large files.
Using hf cli, you can specify cache location and force to not use symlinks, but then you use double the space for the models.