Inside the Token classification pipeline (PyTorch)

Course
1

Hi,
I have been looking through one of the part 2 course tutorials for Inside the Token classification pipeline (PyTorch) illustrated by @sgugger. I have tried to use the example in (https://colab.research.google.com/github/huggingface/notebooks/blob/master/course/videos/token_pipeline_pt.ipynb) and used sshleifer/tiny-dbmdz-bert-large-cased-finetuned-conll03-english as a model-checkpoint. However, I had the following error,
AttributeError: ‘list’ object has no attribute ‘argmax’ in `import torch

probabilities = torch.nn.functional.softmax(outputs.logits, dim=-1)[0].tolist()
predictions = probabilities.argmax(dim=-1)[0].tolist()
print(predictions)`

Am I missing something?

Best,
Ghadeer

2

You are converting the tensor to a Python list which does not have an argmax function. Try it like this:

probabilities = torch.nn.functional.softmax(outputs.logits, dim=-1)[0]
predictions = probabilities.argmax(dim=-1)[0].tolist()
print(predictions)

Which basically delays list conversion until the very end.

3

Thanks, but how to fetch the results,

Capture
Capture1192×630 71.4 KB

if the predictions variable isn’t itertable?

4

The code snippet from before looks at the prediction for the first token, which makes predictions an int rather than an iterable list. You can get predictions for all the tokens by changing it like this:

probabilities = torch.nn.functional.softmax(outputs.logits, dim=-1)[0]
predictions = probabilities.argmax(dim=-1).tolist()
print(predictions)
5

Yes, I got it. Thanks! However, the final output of this piece of code doesn’t perform as expected. Here is the link to the code.

Capture2
Capture2777×632 56.4 KB

6

You are welcome, I don’t know about the last problem. My output looks like this:

image
image970×488 18.2 KB

7

Yes, same for me. In the end, we need to merge the B- and I- labels entities. @sgugger any clue?

8

Hey @ghadeermobasher there are several strategies that you can use to merge the entities and my suggestion would be to inspect the implementation in the token-classification pipeline. As you can see in the docstring there are quite some subtleties associated with merging entities correctly (depends on the language, tokenizer etc)!

1 Like