Hi! I am trying to recover the initial input representation that BERT generates for each token in a given sequence. I know this initial representation is made by adding the token embedding with the positional and the segment embedding (see picture below from the paper). I also know these individual …

How to get a model's initial input representation?

yabramuvdi June 21, 2022, 8:24am 3

Hi Derrick! Great idea. Thanks a lot for the recommendation. I will implement this.

Topic		Replies	Views
What's the input of BERT? Beginners	4	9761	December 23, 2022
What should be used as sentence embedding for BertModel? Beginners	0	1911	May 24, 2021
How to get [CLS] embeddings from BertForTokenClassification model Beginners	3	15235	November 27, 2023
Bert embedding layer Beginners	1	2750	January 4, 2022
Accessing uncontextualized BERT word embeddings Beginners	2	1504	October 30, 2020