I’ve been unsuccessful in freezing lower pretrained BERT layers when training a classifier using Huggingface. I’m using AutoModelForSequenceClassification particularly, via code below, and I want to freeze the lower X layers (ex: lower 9 layers). Is this possible in HuggingFace, and if so what code …

Freeze Lower Layers with Auto Classification Model

Morphe May 25, 2023, 1:30am 7

Topic		Replies	Views
Gradual Layer Freezing with huggingface model 🤗Transformers	1	713	February 10, 2021
How to freeze some layers of BertModel Beginners	8	15457	August 25, 2022
How to freeze layers using trainer? Beginners	10	24128	July 14, 2021
Save a Bert model with custom forward function and heads on Hugginface Intermediate	1	1671	June 7, 2022
BERT finetuning "index out of range in self" Intermediate	2	3875	August 24, 2021