Git push rejected

🤗Datasets
1

Hi,

I am trying to create an Audio dataset. Not able to push data to my repository. Getting error like as shown below.

Uploading LFS objects: 100% (23938/23938), 28 GB | 0 B/s, done.                                                                               
Enumerating objects: 25039, done.
Counting objects: 100% (25039/25039), done.
Delta compression using up to 32 threads
Compressing objects: 100% (24983/24983), done.
Writing objects: 100% (25038/25038), 11.29 MiB | 297.00 KiB/s, done.
Total 25038 (delta 67), reused 24963 (delta 46), pack-reused 0
remote: Resolving deltas: 100% (67/67), done.
remote: Scanning LFS files for validity...
remote: KeyError: 'data'
To https://huggingface.co/datasets/my_dataset
 ! [remote rejected]   main -> main (pre-receive hook declined)

Audio files and csvs are within ‘data’ folder. Please help me to resolve this.

2

hi @anjalyjayakrishnan ,

It looks like you’re trying to push to https://huggingface.co/datasets/my_dataset ?

Could you please share your .git/config inside your repo, if it does not contain any confidential tokens?
You should see something similar to this

[core]
	repositoryformatversion = 0
	filemode = true
	bare = false
	logallrefupdates = true
	ignorecase = true
	precomposeunicode = true
[remote "origin"]
	url = https://huggingface.co/datasets/radames/test
	fetch = +refs/heads/*:refs/remotes/origin/*
[branch "main"]
	remote = origin
	merge = refs/heads/main
[lfs]
	repositoryformatversion = 0
[lfs "https://huggingface.co/datasets/radames/test.git/info/lfs"]
	locksverify = false
	access = basic
3

Hi @radames ,

Thanks for your time!

Here is my .git/config content.

[core]
        repositoryformatversion = 0
        filemode = true
        bare = false
        logallrefupdates = true
[remote "origin"]
        url = https://huggingface.co/datasets/bridgeconn/snow_mountain
        fetch = +refs/heads/*:refs/remotes/origin/*
[branch "main"]
        remote = origin
        merge = refs/heads/main
[lfs "customtransfer.multipart"]
        path = huggingface-cli
        args = lfs-multipart-upload
[credential]
        helper = store
[lfs "https://huggingface.co/datasets/bridgeconn/snow_mountain.git/info/lfs"]
        locksverify = false
        access = basic
[pull]
        rebase = false
4

hi, it seems all right on your git/config. Do you have more context on the original error? maybe more lines about why your push was rejected?

Also noticed here .gitattributes · bridgeconn/snow_mountain at main
that csv files are not on your list, do you have large csv files > 10MB ? if that’s the case you have to use lfs to track them

git lfs track "*.csv"
git add *.csv
git commit
5

@radames ,

My csvs are of < 10MB only. My ‘data’ folder is of size 27GB. It has audios and csvs. This is what i gets while doing git push

git push origin main
Uploading LFS objects: 100% (23938/23938), 28 GB | 0 B/s, done.                                                                               
Enumerating objects: 25048, done.
Counting objects: 100% (25047/25047), done.
Delta compression using up to 32 threads
Compressing objects: 100% (24989/24989), done.
Writing objects: 100% (25044/25044), 11.23 MiB | 295.00 KiB/s, done.
Total 25044 (delta 72), reused 24963 (delta 46), pack-reused 0
remote: Resolving deltas: 100% (72/72), completed with 1 local object.
remote: Scanning LFS files for validity...
remote: KeyError: 'data'
To https://huggingface.co/datasets/bridgeconn/snow_mountain
 ! [remote rejected]   main -> main (pre-receive hook declined)
error: failed to push some refs to 'https://huggingface.co/datasets/bridgeconn/snow_mountain'
6

can you try to track the *.csv via lfs? git lfs track "*.csv" you might need to restart the repo and force push

7

@radames ,

Sorry for the late reply. Tracked csv files… still facing the same issue. Do you have any ideas that I can try?

8

hi @anjalyjayakrishnan sorry it’s still not working,
is this the repo you’re trying? I don’t see csv tracked on .gitattributes here

Maybe @mariosasko can help us?

9

Yes it is the repo. I tried tracking csvs and few more things. But I couldn’t push my files. Then started again from my initial commit (that’s why csv not in the .gitattributes) and tried to push few audios, csv, and loading script. It worked and you can see that in the repo. Not able to push full data.

10

ok, I’ve cloned you repo and restart the commits, can you try cloning or duplicating radames/snow-mountain-test · Datasets at Hugging Face
and then add, commit and push? Or share a subset of your dataset?

11

@radames ,

Couldn’t push data to radames/snow-mountain-test. I can share a part of my dataset. How can I share it?

12

@radames ,

Thanks a lot for your time!

I managed to push the data. I have sub-directories in my audio folder. Just tried to push by making them zip files. It worked.

13

This is happening to me too, don’t know how to fix it

Uploading LFS objects: 100% (10000/10000), 8.5 GB | 21 MB/s, done.
Enumerating objects: 10014, done.
Counting objects: 100% (10014/10014), done.
Delta compression using up to 16 threads
Compressing objects: 100% (10013/10013), done.
Writing objects: 100% (10014/10014), 1.37 MiB | 789.00 KiB/s, done.
Total 10014 (delta 0), reused 10014 (delta 0), pack-reused 0
remote: Scanning LFS files for validity...
remote: KeyError: 'data'
To https://huggingface.co/datasets/stma/danbooru-metadata
 ! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to 'https://huggingface.co/datasets/stma/danbooru-metadata'
14

Gave up and tried disabling LFS, now I get this error when trying to push:

Enumerating objects: 24029, done.
Counting objects: 100% (24029/24029), done.
Delta compression using up to 16 threads
Compressing objects: 100% (24027/24027), done.
error: RPC failed; HTTP 408 curl 22 The requested URL returned error: 408
send-pack: unexpected disconnect while reading sideband packet
Writing objects: 100% (24028/24028), 7.00 GiB | 20.13 MiB/s, done.
Total 24028 (delta 0), reused 24028 (delta 0), pack-reused 0
fatal: the remote end hung up unexpectedly
Everything up-to-date
15

Tried again, this time making a single commit for each directory of files. They’re 12 directories of 1000 zip files each. That worked. Not sure why it would not work if I committed them all at once. It’s 11999 files total

16

Hello,

We have investigated the problem, and we have deployed a fix for it, let me know if happens again

Cheers

1 Like
17

@stma ,

I just saw your previous messages. Sorry for this late reply. Hope your issue is fixed.