I am trying to freeze some layers of my masked language model using the following code:
for param in model.bert.parameters(): param.requires_grad = False
However, when I execute the code above, I get this error:
AttributeError: 'RobertaForMaskedLM' object has no attribute 'bert'
In my code, I have the following imports for my masked language model, but I am unsure what is causing the error above:
from transformers import AutoModelForMaskedLM model = AutoModelForMaskedLM.from_pretrained(model_checkpoint)
So far, I have tried to replace
model in my code, but that did not work.
Any help would be good.