Sorry for the issue, I don’t really write any code but only use the example code as a tool.
I trained with my own NER dataset with the transformers example code.
I want to get sentence embedding from the model I trained with the token classification example code here (this is the older version of example code by the way.)
I want to get the sentence embedding from the trained model, which I think the [CLS] token embedding output should be one way.
This github issue answer answers exactly how to get an embedding from a BertModel (I can also get [CLS] token as the first token in sentence)
The answer code is copy paste below:
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased') model = BertModel.from_pretrained('bert-base-uncased') input_ids = torch.tensor(tokenizer.encode("Hello, my dog is cute")).unsqueeze(0) # Batch size 1 outputs = model(input_ids) last_hidden_states = outputs # The last hidden-state is the first element of the output tuple
So here comes my problem: How to get embeddings from BertForTokenClassification instead of BertModel? Can I simply replace the BertModel with BertForTokenClassification in the code and the expected output is what I wanted?