[HELP] NER task single sentence/sample prediction

🤗Transformers
1

Hello everybody.

I am trying to predict with the NER model, as in the tutorial from huggingface (it contains only the training+evaluation part).

I am following this exact tutorial here : https://github.com/huggingface/notebooks/blob/master/examples/token_classification.ipynb.

It works flawlessly, but the problems that I have begin when I try to predict on a simple sample.

loaded_model = AutoModel.from_pretrained('./my_model_own_custom_training.pth', from_tf=False)



input_sentence = "John Nash is a great mathematician, he lives in France"
tokenized_input_sentence = tokenizer([input_sentence],
                                     truncation=True, 
                                     is_split_into_words=False,
                                     return_tensors='pt')
predictions = loaded_model(tokenized_input_sentence["input_ids"])[0]

predictions is of shape (1,13,768)

How can I arrive at the final result of the form [JOHN ↔ ‘B-PER’, … France ↔ “B-LOC”], where B-PER and B-LOC are two ground truth labels, representing the tag for a person and location respectively?

The result of the prediction is:

torch.Size([1, 13, 768])

If I write:

print(predictions.shape)
print(predictions.argmax(axis=2))
tensor([613, 705, 244, 620, 206, 206, 206, 620, 620, 620, 477, 693, 308])

I get the tensor above.

However I would have expected to get the tensor representing the ground truth [0…8] labels from the ground truth annotations.

Summary when loading the model :

loading configuration file ./my_model_own_custom_training.pth/config.json
Model config DistilBertConfig {
“name_or_path": “distilbert-base-uncased”,
“activation”: “gelu”,
“architectures”: [
“DistilBertForTokenClassification”
],
“attention_dropout”: 0.1,
“dim”: 768,
“dropout”: 0.1,
“hidden_dim”: 3072,
“id2label”: {
“0”: “LABEL_0”,
“1”: “LABEL_1”,
“2”: “LABEL_2”,
“3”: “LABEL_3”,
“4”: “LABEL_4”,
“5”: “LABEL_5”,
“6”: “LABEL_6”,
“7”: “LABEL_7”,
“8”: “LABEL_8”
},
“initializer_range”: 0.02,
“label2id”: {
“LABEL_0”: 0,
“LABEL_1”: 1,
“LABEL_2”: 2,
“LABEL_3”: 3,
“LABEL_4”: 4,
“LABEL_5”: 5,
“LABEL_6”: 6,
“LABEL_7”: 7,
“LABEL_8”: 8
},
“max_position_embeddings”: 512,
“model_type”: “distilbert”,
“n_heads”: 12,
“n_layers”: 6,
“pad_token_id”: 0,
“qa_dropout”: 0.1,
“seq_classif_dropout”: 0.2,
“sinusoidal_pos_embds”: false,
"tie_weights”: true,
“transformers_version”: “4.8.1”,
“vocab_size”: 30522
}

2

cc’ing @sgugger, these demo notebooks really need an inference part.

To do inference with NER, you need to load an AutoModelForTokenClassification rather than an AutoModel, like so:

from transformers import AutoTokenizer, AutoModelForTokenClassification

tokenizer = AutoTokenizer.from_pretrained("path_to_directory")
model = AutoModelForTokenClassification.from_pretrained("path_to_directory")

input_sentence = "John Nash is a great mathematician, he lives in France"
encoding = tokenizer([input_sentence], return_tensors='pt')

# forward pass
outputs = model(**encoding)
logits = outputs.logits

predictions = logits.argmax(-1)

The logits will be of shape (batch_size, seq_len, num_labels).

1 Like
3

Wow, it works perfectly.

Thank you for the awesome response!!!

1 Like