Push_to_hub_callback doesn't work

🤗Hub
1

Im using push to hub callback but it doesn’t work as it’s suppose to, it only creates new repo but doesn’t push anything after each epoch and when all epochs are done training just keeps going I need to manuali interrupt it. I tried puting my token directly into push_to_hub_callback but it changes nothing

2

@Gozdi Hello, can you post your code here and also the repository?

3

@merve
here is colab notebook I used for training

and here is repository https://huggingface.co/Gozdi/electra-base-finetuned-coqa
but i created it manually

4

@Gozdi we’re investigating the issue :hugs:

5

I am facing the exact same issue @Gozdi mentioned. My training arguments are following. I am already using notebook_login interface to provide token. Surprisingly, everything works seamless except the push_to_hub part. I am noticing that it is saying several commits will be pushed upstream, but nothing shows up in the hub. This is happening in google colab and I am storing my model in mounted google drive (maybe that could be a trouble?).

At the end I used used in this tutorial to push the model to hub manually. Happy to share more info if required.

args = TrainingArguments(
    output_dir='/content/drive/MyDrive/'+folder_name,
    evaluation_strategy="epoch",
    save_total_limit = 2, 
    save_strategy="epoch",
    learning_rate=learning_rate,
    metric_for_best_model='f1',
    greater_is_better=True, 
    load_best_model_at_end=True,
    num_train_epochs=num_epochs,
    weight_decay=0.01,
    logging_steps=100,
    hub_private_repo=True,
    push_to_hub=True,
    hub_model_id=folder_name
    )
6

It does not work for me either, but I found a way around, here is how I do it:

I define my push to hub call back as normal and train my model, it saves everything in the output directory and when the training is finished, it will produce the below error and I just ignore it:


SError: EOF
error: failed to push some refs to 'https://huggingface.co/your-user-name/your-name-of-the-model'

Then, I use the below code to manually upload it:


from huggingface_hub import HfApi

api = HfApi()

api.upload_folder(
    folder_path = "path-to-your-output-directory",
    repo_id = "hf-username/" + name-of-the-model,
    repo_type = "model",
)

It works for me.