Trainer.push_to_hub is taking lot of time, is this expected behaviour?

Beginners
1

Hi, I’m trying to push my model to HF hub via trainer.push_to_hub and I see that it is taking a lot of time. Is this the expected behaviour?

below are the o/p

Saving model checkpoint to xlm-roberta-base
Configuration saved in xlm-roberta-base/config.json
Model weights saved in xlm-roberta-base/pytorch_model.bin
tokenizer config file saved in xlm-roberta-base/tokenizer_config.json
Special tokens file saved in xlm-roberta-base/special_tokens_map.json
Several commits (2) will be pushed upstream.
The progress bars may be unreliable.

2

@sgugger can you please help me out with this error?

3

What error are you seeing? Pushing to the hub may take time if you have an unreliable connection, it all depends on the internet speed.

4

Thank you, I figured that the error occurs when I mount the drive(maybe that’s when push_to_hub causes an issue), now that I keep the data in collab itself, it’s no longer an issue.

3 Likes
5

Thanks for this, spent hours trying to push to hub from colab drive and was not working. maybe something to flag as part of the course.

6

Can you please explain what you mean by this? How would I do this with Image data?

7

I also ran into this problem. It resolved itself when I deleted some of the models that I had previously pushed to hub. Hope it can help someone in the future.

8

That’s a good catch! thanks for sharing your findings and helping to identify the source of the problem.