Fine-tune BERT and Camembert for regression problem

valhalla · July 17, 2020, 4:35am

Hi @sundaravel, you can check the source code for BertForSequenceClassification here. It also has code for regression problem.

Specifically for regression your last layer will be of shape (hidden_size, 1) and use MSE loss instead of cross entropy

Topic		Replies	Views
BERT (CamemBERT) for Sequence Classification maps any sequence to the exact same encoding Models	0	205	July 7, 2023
Fine Tune BERT Models Beginners	5	16535	June 25, 2021
Finetuning Bert to adapt to the newly added class 🤗Transformers	0	81	June 22, 2024
Questions about my first code on fine-tuning BERT model for text-classification Beginners	0	1507	April 26, 2022
Sentence Embeddings From Fine-Tuned BERTForSequenceClassification 🤗Transformers	1	1676	September 29, 2021