How to use `.modules()` command to get all the parameters that pertains to the uppermost layer of `roberta-large` model?

🀗Transformers
1

Hello,
I would like to apply the function f to the parameters that pertains to the 24th layer (the uppermost layer) of the RobertaForMultipleChoice pre-trained model (roberta-large). How should I fix the loop below so that I only fix the parameters that are from the 24th layer? Currently, the loop applies the function f to every paramete in a Transformer.
Thank you,

for m in model_RobertaForMultipleChoice.modules():
     for name, value in list(m.named_parameters(recurse=False)):
         setattr(m, name, f)
2

check the names of modules

for nm in model_RobertaForMultipleChoice.named_modules(): print(nm[0])

choose the module you want

for name, mod in model_RobertaForMultipleChoice.named_modules(): 

  if name==‘the 24 th layer module name’:  mod.parameters()
2 Likes