RuntimeError: CAS service error when pushing large dataset to Hugging Face Hub

Hello! Thanks for the report @mahwizzzz

I don’t believe this is related to this issue, that @John6666 linked to, but is potentially something different. While we look into this could you provide a few details?

The full method signature that you’re using? e.g.:

dataset.push_to_hub("<organization>/<dataset_id>", num_shards=1024, private=True ..... )

Information about your environment - OS/packages & versions/etc (if you have the huggingface-cli installed you can just run huggingface-cli env and provide that). At a minimum knowing what operating system you have and what version of hf-xet is installed would be quite helpful.

If you retry and this continues to be an issue, you can try to upload with hf-xet disabled (export HF_HUB_DISABLE_XET=1) to help unblock you. This will fall back to HTTP upload and you can then unset that environment variable after.

1 Like