Should we normalize the BertModel.forward()

MahdiA · December 23, 2021, 4:46pm

When we use the last_hidden_state of BertModel.forward, do we need to normalize that results for such job like Sentiment Analysis? (Here is a distribution of last_hidden_state over a sample text)

Topic		Replies	Views
Is last_hidden_state the output of Encoder block? Beginners	1	425	December 23, 2021
BertModel.forward() output caveat removed? Models	6	641	September 5, 2020
Should I use BertModel or BertModelForLM? Beginners	2	413	February 10, 2022
Question about last_hidden_state of the bert model Beginners	0	277	December 7, 2023
For tuning a classifier head on a pretrained BERT should I use `last_hidden_state` or `outputs[0][:, 0, :]` from the BERT? Beginners	0	156	February 15, 2024

Should we normalize the BertModel.forward()

Related topics