Chapter 2 questions

Use this topic for any question about Chapter 2 of the course.

  1. In the Handling multiple sequences page of Chapter 2, there is a bug in the code under Attention masks section.

    Page: Using 🤗 Transformers - Hugging Face Course

    The PyTorch toggle is on, but the code uses Tensorflow’s tf.constant function.

  2. There is a typo on https://huggingface.co/course/chapter2/6?fw=pt

  3. Isn’t wordpiece a subword algorithm as well?

Thanks for flagging all of this, will push a fix in the morning!

  • In Chapter 2, under the Putting it all together page, the above code snippet should include padding=True :slight_smile:

  • Under Wrapping up: From tokenizer to model, the last line of code snippet should be changed to output = model(**tokens)

Page: Putting it all together

@sgugger I’m absolutely loving this course! A great refresher to the library with really intuitive videos and tutorials to wade through and understand the Hugging Face Library. I honestly wished I had this resource when I started out. Can’t wait for the next part of the course! :slight_smile:

1 Like

thanks for reporting the bugs and suggested fixes @harish3110! will push a fix this afternoon :slight_smile:

1 Like

I found a small typo in the section Behind the pipeline - Postprocessing the output.

  • error: [0.9946, 0.0544]
  • correction: [0.9995, 0.0005]

The following block gives a little bit more context (see the section Postprocessing the output)

tensor([[4.0195e-02, 9.5980e-01],
        [9.9946e-01, 5.4418e-04]], grad_fn=<SoftmaxBackward>)

Now we can see that the model predicted [0.0402, 0.9598] for the first sentence and [0.9946, 0.0544] for the second one. These are recognizable probability scores.

Thanks for reporting, will fix this morning!

1 Like

In the Handling multiple sequences notebook (for Tensorflow) of Chapter 2, there is a bug in the code under Tokenization section.

No that is not a bug, the course explicitly says this doesn’t work and explains why.

Okay sorry for that, have to go over it again. Thanks!

1 Like

No worries, I understand why you’d be suprised. The notebooks are auto-generated, so I can’t add some comments in Markdown cells but I can add comments in the code!

1 Like