Saving tokenizer's configuration

hi,

As the tokenizer is AutoTokenizer, you can use tokenizer.save_pretrained(your path) to save it.

1 Like