I use the code(https://github.com/shibing624/MedicalGPT/blob/main/pretraining.py) to do pretraining. When I use resume_from_checkpoint to restart the training process from the checkpoint of 2000 step, the learning rate scheduler is as follows.
Same problem! Any progress here?