After loading minilm, if I print the model it still shows as BertModel

🤗Transformers
1

after loading “microsoft/MiniLM-L12-H384-uncased” using AutoModelForSequenceClassification, if I print the model it still shows as BertModel. Is this something expected?

model = AutoModelForSequenceClassification.from_pretrained(“microsoft/MiniLM-L12-H384-uncased”,
num_labels=5,
output_attentions=False,
output_hidden_states=False)
model.cuda()

output:

BertForSequenceClassification(
(bert): BertModel(
(embeddings): BertEmbeddings(
(word_embeddings): Embedding(30522, 384, padding_idx=0)
(position_embeddings): Embedding(512, 384)
(token_type_embeddings): Embedding(2, 384)
(LayerNorm): LayerNorm((384,), eps=1e-12, elementwise_affine=True)
(dropout): Dropout(p=0.1, inplace=False)
)
(encoder): BertEncoder(
(layer): ModuleList(
(0): BertLayer(
(attention): BertAttention(

1 Like