Hi there,
I am interested in fine-tuning the MPNet pre-trained model on a domain-specific corpus. I understand that the MPNet model combines both autoregressive and autoencoding language modeling objectives. Does the run_mlm.py or run_mlm_wwm.py script with the MPNet model as argument trains with both objectives or the masked LM objective as the name of the script implies?