Error with new tokenizers (URGENT!)

[Ogayo]

@denocris

• latest pip version 20.3.3 (On Colab I had installed by default 19 and something);

I resolved it.

1. Uninstalled transformers
2. Installed transformers sentencepiece like this : !pip install --no-cache-dir transformers sentencepiece
3. Use_fast= False like this: tokenizer = AutoTokenizer.from_pretrained(“XXXXX”, use_fast=False)