Hello,
I want to further pre-train a RoBERTa model using a customized set of masked tokens (e.g., masking a percentage of high information words or never masking stopwords). Is there a way to do this?
This post asked a similar question, but there is no answer: