Export the embeddings layer of a pre-trained model as a standalone model

Beginners
1

I would like to import a pre-trained model, extract the embeddings layer and export that as a standalone model. I did something like this:

from transformers import AutoTokenizer, AutoModelForPreTraining

tokenizer = AutoTokenizer.from_pretrained("deepset/gelectra-large")
model = AutoModelForPreTraining.from_pretrained("deepset/gelectra-large")

embedding_model = model.electra.embeddings

embedding_model.save_pretrained("...")

This returns the error that the object has no save_pretrained method. Any ideas how this is done?

One idea that came to mind is to include the extracted embeddings layer in a new model object, but not sure what config to use for its initialisation.

2

Solved. I just used the save methods from PyTorch on the extracted embedding objects.