Hello all, i am trying to create distill version of gelectra-base model. For training a student model optimizer has to be defined, as per paper i used Adam optimizer but the losses are not looking good. So anyone have idea about optimizer.
Hello all, i am trying to create distill version of gelectra-base model. For training a student model optimizer has to be defined, as per paper i used Adam optimizer but the losses are not looking good. So anyone have idea about optimizer.