Git consumes 8GB+ of memory when trying to clone from HF

Beginners
1

This repo: NousResearch/Nous-Hermes-2-Yi-34B at main has 23 files, and no subdirectories. It seems pretty simple to me.

But for some reason, git clone consumes so much memory that the process is killed.

I presume it’s a LFS thing because I’ve never seen git need gigs of memory?!? But even the largest file is around half of the RAM that git is eating up.

Any ideas on how to limit memory usage so it doesn’t die?

2

Update:

Downloading pointers only (with GIT_LFS_SKIP_SMUDGE=1), then doing an immediate lfs pull seems to be working (so far). git consuming 0.7GB of memory instead of 8GB+

GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/NousResearch/Nous-Hermes-2-Yi-34B
cd Nous-Hermes-2-Yi-34B
git lfs pull

Edit: Completed successfully. Not sure this is really a “solution”, but it worked.

3

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.