Hugging Face Forums
Saving tokenizer's configuration
Beginners
tntchung
February 24, 2022, 4:22pm
2
hi,
As the tokenizer is AutoTokenizer, you can use tokenizer.save_pretrained(your path) to save it.
1 Like
show post in topic
Related topics
Topic
Replies
Views
Activity
How to save my tokenizer using save_pretrained?
Beginners
5
29566
August 13, 2021
Issue in loading the saved tokenizer
🤗Tokenizers
1
240
January 4, 2024
Save_pretrained() on tokenizer does not generate a tokenizer.json file
🤗Transformers
3
889
August 19, 2024
Load fine tuned model from local
Beginners
4
10292
October 20, 2020
"How to train a new language model from scratch using Transformers and Tokenizers" possibly requiring an update
Site Feedback
4
2560
November 1, 2022