Is it possible to use a from_pretrained() method to respawn model and tokenizer from my own s3 bucket?

I saved a DistilBertModel and a tokenizer with the help of save_pretrained() method. Now, when I load them locally using from_pretrained(’/path_to_distilbert_model’) everything works fine and as intended.
I need these models to be loaded from my own s3 bucket.
But when I try to load them using TFDistilBertModel.from_pretrained(‘s3://my_bucket/path_to_distilbert_model’) there is an error, stating that model configuration file cannot be found or that ‘vocab.txt’ cannot be found if I use DistilBertTokenizer.from_pretrained().
Is it possible to load a model from my own s3 bucket and what is the suggested way to do that? Thanks

Looking at the source code, I do not think it is possible to load models from external URLs other than the HuggingFace bucket.

That being said, why don’t you upload your model to the HuggingFace model hub?

Wanted to see if you’ve found any solution to this problem?

I’m having the same problem where I need to read a pretrained model via from_pretrained(“s3://bucket_name/model_folder”), which throws the same errors.
I think the reason is, based on the source code, os.path.isdir(“s3://bucket_name/model_folder”) returns False.

Any solution so far?

On S3 there is no such concept as a “folder” link. That could be a reason that providing a folder path is not working.