Trainer and TrainingArguments - gradual unfreezing

Since further pre-training BERT might suffer from catastrophic forgetting, I am wondering, is it possible to use gradual unfreezing somehow with the Trainer module? I could do it with PyTorch and a classical training loop, but I was looking if you already have it implemented in the Trainer class.

or TrainingArguments class.