Learning rate setting

Hey, I wonder is it possible to set different learning rates for different parts in the model ?This is ususlly considered as a trick in bert fine-tuning.

You can pass your own optimizer to the Trainer, in which you can define your parameter groups as you like.