Hey, if I fine tune a BERT model is the tokneizer somehow affected? If I save my finetuned model like: bert_model.save_pretrained(’./Fine_tune_BERT/’) is the tokenizer saved too? is the tokenizer modified? Do I need to save it too? Bercause loading the tokenizer like: tokenizer = BertTokenizer…

Load fine tuned model from local

datistiquo October 20, 2020, 2:11pm 3

But the important issue is, do I need this? Can I still download it the normal way? Is the tokenizer affected by model fientuning? I assume no, so I could still use the tokenizer from your API?

Topic		Replies	Views
Simple Save/Load of tokenizer not working 🤗Transformers	2	1668	November 4, 2020
Saving tokenizer's configuration Beginners	1	2826	February 24, 2022
Does the tokenizer get fine tuned? Beginners	0	275	July 24, 2021
How to save my tokenizer using save_pretrained? Beginners	5	29625	August 13, 2021
Using a BertWordPieceTokenizer trained from scratch from transformers 🤗Tokenizers	2	5064	March 26, 2021

Load fine tuned model from local

Related topics