Hey, if I fine tune a BERT model is the tokneizer somehow affected? If I save my finetuned model like: bert_model.save_pretrained(’./Fine_tune_BERT/’) is the tokenizer saved too? is the tokenizer modified? Do I need to save it too? Bercause loading the tokenizer like: tokenizer = BertTokenizer…

Load fine tuned model from local

datistiquo October 20, 2020, 2:11pm 3

But the important issue is, do I need this? Can I still download it the normal way? Is the tokenizer affected by model fientuning? I assume no, so I could still use the tokenizer from your API?

Topic		Replies	Views
Load pretrained model's tokenizer with or without vocabulary? Beginners	2	164	August 30, 2024
Does the tokenization in BERT change after fine-tuning? Models	0	595	January 27, 2023
Saving tokenizer's configuration Beginners	1	2826	February 24, 2022
Simple Save/Load of tokenizer not working 🤗Transformers	2	1668	November 4, 2020
Does the tokenizer get fine tuned? Beginners	0	275	July 24, 2021

Load fine tuned model from local

Related topics