Hi! I wanted to use automodelformaskedlm
to pretrain a model I built in nn.Module
with MLM objective. I searched a lot but it looks like most tutorials still ask that you use a base model within HF. How can I achieve this? Meanwhile, I also want to compute the per-unique-token mask reconstruction accuracy. Is there a parameter or a tutorial for this? Thanks in advance!