Whether upon trying the inference API or running the code in “use with transformers” I get the following long error: “Can’t load tokenizer using from_pretrained, please update its configuration: Can’t load tokenizer for ‘remi/bertabs-finetuned-extractive-abstractive-summarization’. If you were tryi…

What to do when HuggingFace throws "Can't load tokenizer"

kkedia June 9, 2023, 9:05am 4

@EssamWisam How were you able to solve this problem. I am facing the same issue. The code is working in one of the GPUs but when i try to run it azure gpu i am getting this issue

Topic		Replies	Views
Tokenizer is not being loaded on Huggingface Inference 🤗Tokenizers	0	1000	September 22, 2022
Huggingface inference API issue 🤗Tokenizers	0	511	January 10, 2023
Tokenizer issue in Huggingface Inference on uploaded models Beginners	7	3068	January 9, 2024
Model hub: Can't load tokenizer using from_pretrained 🤗Hub	2	1610	January 5, 2022
Cant load tokenizer using from_pretrained, `use_auth_token=True` error when token is being used Inference Endpoints on the Hub	7	7700	August 6, 2023

What to do when HuggingFace throws "Can't load tokenizer"

Related topics