Is there any way to pre-train a model (i.e. RoBERTa), using my own corpus, but with a custom mask strategy? For instance, I want to look up a dictionary to choose which words to mask.
Is there any way to pre-train a model (i.e. RoBERTa), using my own corpus, but with a custom mask strategy? For instance, I want to look up a dictionary to choose which words to mask.