Error with new tokenizers (URGENT!)

🤗Tokenizers
3

Thank you so much @FL33TW00D. Two steps were required.

  1. latest pip version 20.3.3 (On Colab I had installed by default 19 and something);
  2. use_fast = False.

Working a lot on Italian or personal custom models, all of them were disabled all of a sudden!

2 Likes