I am having trouble understanding how to iterate through the BERT layers in order to freeze/unfreeze them. It is my understanding that BERT models have 12 layers but when I do model.parameters() or model.bert.parameters(), I get>100 iterations. This seems wrong but I don’t know what I should be doing. I am doing this on a BERTForMaskedLM.from_pretrained model.
Thanks for any assistance