Hi,
I trained a custom sense embeddings based on Wordnet definition and tree structure.
Now I want to test the embeddings by fine tuning BERT masked LM so the model predicts the most likely sense embedding. For example:
" I need to go to the [bank] today" → bank.wn.02
I’m uncertain how to accomplish this.
- Can I provide a set of output labels with their embeddings different from the input embedding and input_ids?
- Should I build a custom softmax layer from the output_hidden_states of the BERTModel