There is nothing out of the box in the library to unfreeze parts of your model during training. You can pass the model with some layers frozen, using the code you wrote, but it will stay this way.
You can try to use a TrainerCallback to unfreeze parts of the model in the middle of the training (after a given number of steps/epochs).