Inserting custom layer after embeddings layer in BERT

riyag2805 · June 26, 2023, 9:27am

Hi,

I want to add a fully connected dense layer of dimension 256, right after the initial BERT embedding layer, i.e. between the embedding layer and the first encoder layer. Is there an easier way to go about this, other than making changes in the code of the original BertModel class?

Topic		Replies	Views
New layer in bert embeddings 🤗Transformers	1	637	April 1, 2022
New Layer in BERT 🤗Transformers	0	181	September 25, 2022
Correct way to implement custom model on top of pretrained bert? Beginners	0	732	November 19, 2022
Custom bert embedding cause "RuntimeError: Trying to backward through the graph a second time" Intermediate	0	804	March 10, 2023
How to get a middle layer from a model with custom head on top of bert model Beginners	0	549	July 6, 2022

Inserting custom layer after embeddings layer in BERT

Related Topics