Trainer.push_to_hub is taking lot of time, is this expected behaviour?

Hi, I’m trying to push my model to HF hub via trainer.push_to_hub and I see that it is taking a lot of time. Is this the expected behaviour?

below are the o/p

Saving model checkpoint to xlm-roberta-base
Configuration saved in xlm-roberta-base/config.json
Model weights saved in xlm-roberta-base/pytorch_model.bin
tokenizer config file saved in xlm-roberta-base/tokenizer_config.json
Special tokens file saved in xlm-roberta-base/special_tokens_map.json
Several commits (2) will be pushed upstream.
The progress bars may be unreliable.

@sgugger can you please help me out with this error?

What error are you seeing? Pushing to the hub may take time if you have an unreliable connection, it all depends on the internet speed.

Thank you, I figured that the error occurs when I mount the drive(maybe that’s when push_to_hub causes an issue), now that I keep the data in collab itself, it’s no longer an issue.

3 Likes

Thanks for this, spent hours trying to push to hub from colab drive and was not working. maybe something to flag as part of the course.

Can you please explain what you mean by this? How would I do this with Image data?

I also ran into this problem. It resolved itself when I deleted some of the models that I had previously pushed to hub. Hope it can help someone in the future.

That’s a good catch! thanks for sharing your findings and helping to identify the source of the problem.