How to get a model's initial input representation?

[dblakely]

It doesn’t output the embeddings, but you could make a custom subclass of BertModel that does. For example, you could override this method and subclass the BaseModelOutputWithPoolingAndCrossAttentions class to extract the embeddings.