Value error : sentencepiece

Beginners
1

Hi,

I have value error: This tokenizer cannot be instantiated. Please make sure you have sentencepiece installed in order to use this tokenizer.

I installed and updated sentencepiece (0.1.95) but still getting the same error. Can someone help?
Thank you!

2

Hi @Katarina, what happens if you try installing transformers in a new environment with

pip install transformers[sentencepiece]

Does that solve the problem?

3 Likes
3

Hi, thanks on answer…I tried with this but still the same error

4

Can you please share the code you are running and the full stack trace / error message?

5

Acctually it is working after I restarted my enviroment…thank you!

1 Like
6

This works:

pip install sentencepiece
1 Like
7

use this one-

pip install transformers[sentencepiece] datasets

8

this has to do with you shell actually. You are probably using zsh if you are on Mac, and you are getting zsh: no matches found: transformers[sentencepiece].

Solution: pip install "transformers[sentencepiece]"

4 Likes
9

Fix: pip install transformers[sentencepiece]
if it is installed and still getting error, restart the kernel

2 Likes
10

Kernel Restart solved this problem.

11

Thank you!

12

Use pip install transformers[sentencepiece]

And restart the kernel. It will work :slight_smile: