Custom BERT Masked LM output embedding

Hi,
I trained a custom sense embeddings based on Wordnet definition and tree structure.
Now I want to test the embeddings by fine tuning BERT masked LM so the model predicts the most likely sense embedding. For example:
" I need to go to the [bank] today" → bank.wn.02

I’m uncertain how to accomplish this.

  1. Can I provide a set of output labels with their embeddings different from the input embedding and input_ids?
  2. Should I build a custom softmax layer from the output_hidden_states of the BERTModel