Pretrain bert model using BOTH bert data custom data

[Keyu]

Hello, I wonder if there is a tutorial for incremental training bert model with custom data in pytorch. All I found on website is pretraining from scratch using only custom dataset. I also want to leverage bert dataset if thta’s possible.

Thanks!