While following instructions on Fine-tuning with custom datasets — transformers 4.7.0 documentation using TensorFlow Keras, model fit produces below problem and fail to start training
from transformers import TFAutoModelForQuestionAnswering
model = TFAutoModelForQuestionAnswering.from_pretrained("bert-base-multilingual-cased")
...
model.fit(...)
TypeError: The two structures don't have the same sequence type. Input structure has type <class 'tuple'>, while shallow structure has type <class 'transformers.modeling_tf_outputs.TFQuestionAnsweringModelOutput'>.
I suspect that it is related to formatting output for Keras requirements as below:
# Keras will expect a tuple when dealing with labels
train_dataset = train_dataset.map(lambda x, y: (x, (y['start_positions'], y['end_positions'])))
since the error message is about output and it says tuples are not transformers.modeling_tf_outputs.TFQuestionAnsweringModelOutput.
Help?
My platform is Windows 10 and libraries are
print(tf.__version__)
print(torch.__version__)
print(transformers.__version__)
2.4.0
1.9.0+cu111
4.8.2