Mobilebert, training from scratch. Not seeing where loads the teacher

Hello, I am modifying mobilbert with some bells and whistles but I am not finding how to start the pertaining with the teacher-student way. Could you help me with that?

1 Like

The training script is here. In general, everything linked to distillation is in this folder.

3 Likes

Thanks a lot!