Does starting training from a previous checkpoint reset the learning rate?

🤗Transformers
#1

Hi,

I want to start training a new model by loading a previous model I trained, I want to know what happens to the learning rate in this case – does it start at the learning rate I set, or does it start from the prev learning rate of the checkpoint?

1 Like
#2

Hi!

As far as I have experienced, it continues from the last learning rate number saved in the checkpoint from where you resumed.
It also starts from the last epoch number.