So I am trying to experiment with continual learning. As a part of that, I have to train a model several times. it will be a classification model using transformers.
The easiest way is to save the model and load it each time, but I am using some cloud resources to store it, so it will be a real hassle to do that.
So i want to train in 2-3 incremental stages in the same job that I run.
So basically, after trainer.train() on one dataset, i want to continue training on a different dataset.
For this, I am thinking of creating trainer subclass again, with the previous model and same previous parameters.
My questions
- is that the right way? is there a way without creating new trainer?
- which among trainer.model and trainer.model_wrapped should I use in the new trainer so that the trained parameters are retained?