Continuing Pre Training from Model Checkpoint

If you use

trainer.train(resume_from_checkpoint = True)

The Trainer will load the last checkpoint it can find, so it won’t necessarily be the one you specified. It will also resume the training from there with just the number of steps left, so it won’t be any different from the model you got at the end of your initial Trainer.train.

6 Likes