Chapter 3 questions

Course
172

In terms of code, just list() will fix it.

Why the notebook code fails now

Root cause

The tokenizer expects text inputs like:

  • str
  • list[str]
  • batched text-pair inputs built from those

That is how the tokenizer API is documented today. (Hugging Face)

But dataset column access is no longer best thought of as “always a plain Python list.” Current Datasets docs explicitly say that indexing by column name first returns a Column object. (Hugging Face)

So this line:

tokenizer(raw_datasets["train"]["sentence1"])

can fail because the tokenizer sees a dataset column object, not a plain Python list[str].

Background: why this is confusing

Older Datasets docs described dataset["sentence1"] as returning a Python list of values. (Hugging Face)

Current docs describe column-first indexing as returning a Column object. (Hugging Face)

That difference explains why:

  • older examples could work as written
  • current environments can reject the same code
  • the course notebook looks reasonable, but still breaks in practice

So the real background is API/documentation drift across library versions and implementations, not that your code idea was conceptually wrong. (Hugging Face)

Why this notebook uses two sentences at all

This dataset is MRPC, a sentence-pair classification task. Each example has:

  • sentence1
  • sentence2
  • a label telling whether they are paraphrases

The course explains that for BERT, the correct pair format is:

[CLS] sentence1 [SEP] sentence2 [SEP]

and that token_type_ids distinguish the first sentence from the second. (GitHub)

So even when the notebook first tokenizes sentence1 and sentence2 separately, that is only an introductory step. The actual task is to tokenize the pair together. (GitHub)

Minimal fix for the notebook

The smallest code change is to convert each dataset column to a plain list.

Replace this

tokenized_sentences_1 = tokenizer(raw_datasets["train"]["sentence1"])
tokenized_sentences_2 = tokenizer(raw_datasets["train"]["sentence2"])

With this

tokenized_sentences_1 = tokenizer(list(raw_datasets["train"]["sentence1"]))
tokenized_sentences_2 = tokenizer(list(raw_datasets["train"]["sentence2"]))

And replace this:

tokenized_dataset = tokenizer(
    raw_datasets["train"]["sentence1"],
    raw_datasets["train"]["sentence2"],
    padding=True,
    truncation=True,
)

with this:

tokenized_dataset = tokenizer(
    list(raw_datasets["train"]["sentence1"]),
    list(raw_datasets["train"]["sentence2"]),
    padding=True,
    truncation=True,
)

Why this works

list(...) converts the dataset column into the kind of container the tokenizer is documented to accept: a plain list[str]. (Hugging Face)

173

Replace "glue" with "nyu-mll/glue" .

174

I was using datasets 5.0.0 and the error said:

raise HfUriError(uri=raw, msg=f"Repository id must be 'namespace/name', got '{repo_id}'.")
huggingface_hub.errors.HfUriError: Invalid HF URI 'hf://datasets/glue@bcdcba79d07bc864c1c254ccfcedcce55bcc9a8c/.huggingface.yaml'. Repository id must be 'namespace/name', got 'glue'.
175

Bug: Code contradicts the text in the “Erratic Learning Curves” section

Section: Learning Curves — Erratic Learning Curves (Solutions)

Issue: The text states: “In the sample below, we lower the learning rate and increase the batch size.”

However, the accompanying code diff does the opposite for the learning rate — it increases it instead of lowering it:

-learning_rate=1e-5,
+learning_rate=1e-4,

Since 1e-4 (0.0001) is 10x larger than 1e-5 (0.00001), this change raises the learning rate, contradicting the stated goal of stabilizing an erratic loss curve by lowering it.

Suggested fix:

-learning_rate=1e-4,
+learning_rate=1e-5,

Additional note (optional, separate section): The illustration in the “Underfitting” section also appears to be a duplicate of the “Overfitting” section’s image (training loss drops far below validation loss), rather than showing both curves remaining high as described in the text. Might be worth double-checking that image as well.