Hi I used XLM-RoBERTa for fine tuning this model to determine the text language. After training, I uploaded the model to the huggingface repository. When trying to use the Inference API on a model page for a model test I get an error Can’t load tokenizer using from_pretrained, please update its co…

[image] dilolo: XLM-RoBERTa Could you please try the XLMRobertaTokenizer tokenizer instead of AutoTokenizer and let me know?

Model Inference API error

dilolo May 6, 2024, 5:50pm 6

@asif00 Asif thank you very much. I really should have used the XLMRobertaTokenizer
Thanks

1 Like

Topic		Replies	Views
Can't load tokenizer using from_pretrained, Inference API 🤗Tokenizers	4	1803	May 6, 2024
Huggingface inference API issue 🤗Tokenizers	0	510	January 10, 2023
OSError: Can't load tokenizer for 'facebook/xmod-base' 🤗Tokenizers	1	1231	October 6, 2023
Tokenizer is not being loaded on Huggingface Inference 🤗Tokenizers	0	994	September 22, 2022
Inference API Web Widget for tons of public models: Can't load tokenizer using from_pretrained Models	0	350	December 12, 2023