Hi I used XLM-RoBERTa for fine tuning this model to determine the text language. After training, I uploaded the model to the huggingface repository. When trying to use the Inference API on a model page for a model test I get an error Can’t load tokenizer using from_pretrained, please update its co…

Model Inference API error

asif00 May 6, 2024, 3:16pm 5

Could you please try the XLMRobertaTokenizer tokenizer instead of AutoTokenizer and let me know?

1 Like

Topic		Replies	Views
Can't load tokenizer using from_pretrained, Inference API 🤗Tokenizers	4	1803	May 6, 2024
Huggingface inference API issue 🤗Tokenizers	0	510	January 10, 2023
OSError: Can't load tokenizer for 'facebook/xmod-base' 🤗Tokenizers	1	1231	October 6, 2023
Tokenizer is not being loaded on Huggingface Inference 🤗Tokenizers	0	994	September 22, 2022
Inference API Web Widget for tons of public models: Can't load tokenizer using from_pretrained Models	0	350	December 12, 2023