Hey, I am trying to figure out how to freeze layers of a model and read that I had to use for param in model.base_model.parameters(): param.requires_grad = False if I wanted to freeze the encoder of a pretrained MLM for example. But how do I use this with the Trainer? I tried the following: …

How to freeze layers using trainer?

sgugger March 19, 2021, 4:37pm 7

You should inspect the weights to see where they difer then. Trainer will not change the requires_grad value of your parameters.

2 Likes

Freeze Lower Layers with Auto Classification Model

Topic		Replies	Views
How to freeze some layers of BertModel Beginners	8	15432	August 25, 2022
The point of using pretrained model if I don't freeze layers Beginners	1	4457	May 31, 2023
Gradual Layer Freezing with huggingface model 🤗Transformers	1	711	February 10, 2021
Freeze Lower Layers with Auto Classification Model 🤗Transformers	6	13906	May 25, 2023
Training on Domain specific Dataset Beginners	3	587	March 22, 2021