Always only a single Linear layer as the classification head?

incameet · February 23, 2023, 5:24am

For example, in the class BertForSequenceClassification definition, only one Linear layer is used as the classifier. If just one Linear layer is used, doesn’t it just do linear project for pooled_out? Will such a classifier produce good predictions? Why not use multiple Linear layers? Does transformers offer any option for using multiple Linear layers as the classification head?

Topic		Replies	Views
BertForSequenceClassification classification head question 🤗Transformers	0	254	July 7, 2022
Classification Heads in BERT and DistilBERT for Sequence Classification Research	2	974	May 13, 2021
Current best practice for final linear classifier layer(s)? Beginners	3	1918	September 12, 2020
What is the classification head doing exactly? 🤗Transformers	15	17675	April 14, 2023
How do I do multi Class (multi head) classification? 🤗Transformers	6	3489	October 18, 2022

Always only a single Linear layer as the classification head?

Related Topics