Tutorial: Fine-tuning with custom datasets – sentiment, NER, and question answering

🤗Transformers
5

Thanks a lot for creating the tutorial @joeddav!

I ran into an issue with tokenizer. It seems like I cannot just pass my list of texts to tokenizer like in the tutorial. Am I doing something wrong?

tokenizer = DistilBertTokenizerFast.from_pretrained('distilbert-base-uncased')
train_encodings = tokenizer(train_texts, truncation=True, padding=True)
val_encodings = tokenizer(test_texts, truncation=True, padding=True)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-77-d270a8ea6e50> in <module>
----> 1 train_encodings = tokenizer(train_texts, truncation=True, padding=True)
  2 val_encodings = tokenizer(test_texts, truncation=True, padding=True)

TypeError: 'DistilBertTokenizerFast' object is not callable