Git LFS fail for Spaces with Github Workflow

Spaces
1

I am trying to setup Github Workflow for a Hugging Face space.
But I receive following error message:

remote: -------------------------------------------------------------------------        
remote: Your push was rejected because it contains binary files.        
remote: Please use https://git-lfs.github.com/ to store binary files.        
remote: See also: https://hf.co/docs/hub/adding-a-model#uploading-your-files        
remote: -------------------------------------------------------------------------        
remote: Offending files:        
remote:  - ballon.jpg (ref: refs/heads/main)        
remote:  - fountain.jpg (ref: refs/heads/main)        
remote:  - wolf.jpg (ref: refs/heads/main)        
To https://huggingface.co/spaces/satpalsr/RegNet-Image-Classification
 ! [remote rejected] main -> main (pre-receive hook declined)
error: failed to push some refs to 'https://huggingface.co/spaces/satpalsr/RegNet-Image-Classification'
Error: Process completed with exit code 1.

Modified Code I tried:

name: Sync to Hugging Face hub
on:
  push:
    branches: [main]

  # to run this workflow manually from the Actions tab
  workflow_dispatch:

jobs:
  sync-to-hub:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
        with:
          fetch-depth: 0
          lfs: true
      - name: Checkout LFS objects
        run: git lfs checkout
      - name: Push to hub
        env:
          HF_TOKEN: ${{ secrets.HF_TOKEN }}
        run: git push --force https://satpalsr:$HF_TOKEN@huggingface.co/spaces/satpalsr/RegNet-Image-Classification main
1 Like
2

Hey there! As per the error message, the binary files should be tracked with git lfs.

What you can do is

# Track the files being rejected
git lfs track "your_file" # or *.your_extension

# Track things 
git lfs install

Note that if you have commits with those files, it will still be rejected since the binary files are in previous commits without git lfs, so you will need to change that.

2 Likes
3

Not working!
Added these lines & even created a new test space.
Though the .jpg files are tracked successfully, the exact same error message repeats.

To reproduce:

  1. Create a test space at Hugging Face.
  2. Create a repository on Github.
  3. Add a secret HF token in the Github repo.
  4. Add some image files to the Github repo.
  5. Set up the GitHub action workflow.
4

if the git repo is public, can you zip the whole repo (including .git) and share it somewhere? Thanks!

5

Hey @julien-c , here is what you need:
Github Repo: https://github.com/satpalsr/HF-workflow
Github repo zip (including .git) : Check DM
Space Link: https://huggingface.co/spaces/satpalsr/workflow-test

6

I face the same issue as well, I had previous commits to binary files without lfs commits but later on, I enabled git lfs but still my workflow fails since the file is there in the previous commits. How do I fix this?

7

Nevermind, found an answer to remove git history of all large file by running the following git command

git filter-branch --index-filter --f "git rm -rf --cached --ignore-unmatch “PATH TO FILE” HEAD
git push origin --force --all

2 Likes
8

arghh, i am running into the same issue…

Is there any way to avoid rewriting the commit history (which is what git filter-branch will do iiuc).

(Btw, i think there is a " missing on your command at the end @Clatonh )

9

same issue here! track the files but keep the same error

10

hi @joselobenitezg ,
You probably have the offending files committed to the git history without LFS, it’s a common issue. Please reset your commit history, removing the commits with files without LFS tracking.

11

You are right! Solved now!

1 Like
12

Hi I am getting similar error. I added this lines of code in yaml file still not working.

      - name: LFS Install
        run: git lfs install
      - name: LFS Track
        run: git lfs track *.mp4
      - name: Checkout LFS objects
        run: git lfs checkout

still showing following error.

Screenshot 2024-01-04 at 7.39.09 PM
Screenshot 2024-01-04 at 7.39.09 PM1588×552 80.9 KB

13

hi @KushwanthK ,

The message states that your repo has a .mp4 file commit to the git history, please make sure you remove it from the history

then track it with git lfs track "*.mp4"

14

yes thanks
just added this in yaml and it works.

 - name: reset commit
        run: git filter-branch --force --index-filter "git rm --cached --ignore-unmatch output_sample.mp4" --prune-empty --tag-name-filter cat -- --all

reference followed this.

2 Likes
15

After I undid my last commit and followed your instructions, I was able to add the new staged changes to the staging area, committing, and then pushing without a problem. Thank you.

16

Just had the same issue with a .sqlite3 file.

Decided to make a new space, then clone it, edit .gitattributes (already provided by HF) to add the file-format there.

*.sqlite3 filter=lfs diff=lfs merge=lfs -text

Running git lfs track "*.sqlite3" confirmed that it was already tracked with lfs there.
Then push.
Then add all my stuff for the project from a different folder - push - works.

1 Like
17

Below is content of my yml file on Github Workflow :

name: Sync to Hugging Face hub
on:
  push:
    branches: [main]

  # to run this workflow manually from the Actions tab
  workflow_dispatch:

jobs:
  sync-to-hub:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
        with:
          fetch-depth: 0
          lfs: false

      - name: Ignore large files
        run : git filter-branch --index-filter 'git rm -rf --cached --ignore-unmatch "research/codebase/3_vectorstore/db_faiss/index.faiss"' HEAD

    
      - name: Push to hub
        env:
          HF_TOKEN: ${{ secrets.HF_TOKEN }}
        run: git push --force https://rahul-bhoyar-1995:$HF_TOKEN@huggingface.co/spaces/KaggleGPT/KaggleGPT main

But I am still getting this error.


Screenshot 2024-05-17 at 11.28.06 PM