Can't load tokenizer with added special tokens

Hi! I have an annoying issue that I can’t find answers to anywhere online…
Hope someone can help.

I am adding a few special tokens to a gpt2 tokenizer using the following code:

model_name = “gpt2”
model = AutoModelForCausalLM.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)

special_tokens = {
‘eos_token’: “<|endoftext|>”,
‘bos_token’: “<|startoftext|>”,
‘additional_special_tokens’: ["<|speaker1|>", “<|speaker2|>”]

tokenizer.pad_token = tokenizer.eos_token
vocab = tokenizer.get_vocab()

I later save the tokenizer using:

model_save_name = ‘SARC_gpt2_prefinetune_2.0’
tokenizer.save_pretrained(F"/content/drive/MyDrive/Colab Notebooks/saved_models/{model_save_name}")

But when I load the model in a different script using:

tokenizer_1 = AutoTokenizer.from_pretrained(’/content/drive/MyDrive/Colab Notebooks/saved_models/SARC_gpt2_prefinetune_2.0’)

I get this error:

AssertionError: Non-consecutive added token ‘<|startoftext|>’ found. Should have index 50260 but has index 50257 in saved vocabulary.

Does anyone know what I’m doing wrong here?
Thanks in advance