Hi everyone,
I’m Darshan Hiranandani, working on fine-tuning the ALBERT model and my professor has suggested that I either freeze the early layers or refrain from freezing the final layer of the base model. I’m wondering if it’s actually possible to freeze certain layers in ALBERT when fine-tuning, especially in TensorFlow.
From what I know, ALBERT uses parameter sharing across its layers, which raises a concern that freezing specific layers might not be straightforward or even feasible. Has anyone attempted to fine-tune ALBERT with layer freezing in TensorFlow? If so, how did you handle the parameter sharing issue, or did you approach this differently?
I’d appreciate any insights or experiences you can share!
Thanks!
Regards
Darshan Hiranandani