Adding tokens to mT5, tensorflow get ValueError

🤗Transformers
1

I am trying to add new tokens to mT5 model this way:

import transformers import MT5Tokenizer, TFMT5ForConditionalGeneration

tokenizer = MT5Tokenizer.from_pretrained('google/mt5-small')
model = TFMT5ForConditionalGeneration.from_pretrained('google/mt5-small')

tokenizer.add_tokens(['my_new_token_1'])

model.resize_token_embeddings(len(tokenizer))

But get the error:

ValueError: Attempt to convert a value (None) with an unsupported type (<class 'NoneType'>) to a Tensor.

What am I doing wrong?

Same code with torch-version of model works well.