Here is the output of the pre-trained BERT, model.layers =
[<transformers.models.bert.modeling_tf_bert.TFBertMainLayer at 0x7f1d5628ac90>, <tensorflow.python.keras.layers.core.Dropout at 0x7f1d51c04cd0>, <tensorflow.python.keras.layers.core.Dense at 0x7f1d51c04f50>] .
Is there a way to access layers in TFBertMainLayer? In particular, I would like to try a particular regularization method for the tokens Embedding() layer, i.e. I would like to access TFBertEmbeddings and the embedding layer that is self.token_type_embeddings = tf.keras.layers.Embedding(
config.type_vocab_size,
config.hidden_size,
embeddings_initializer=get_initializer(self.initializer_range),
name=âtoken_type_embeddingsâ,
)
with the purpose of setting âembeddings_regularizer=â to a specific value.