Large LFS usage despite super_squash_history

I have a Huggingface repo that I use as scratchspace during training. Intermediate checkpoints are saved here. I delete most of them after training is good.

Since a lot of safetensor files get written there, it uses a massive amount of LFS space. To reduce the storage use I ran api.super_squash_history to collapse the repo into a single commit. However, the storage usage has not shrunk, despite waiting for several days.

What is the correct action here?

1 Like

There used to be a bug where the amount of use wouldn’t decrease even if you squashed it, but I think that’s been fixed. So, is it a bug that’s come back…?

The most reliable thing to do is to download the repo file, delete the repo, create a repo, and upload it again, but that’s a last resort.

1 Like

Thanks!

Even after squashing the total size should be in the 10s of gigabytes. I would rather not download and reupload if possible.

1 Like