I used the pre-trained weights of the DistilBERT model in order to train a text classification model with the SST-2 dataset. Now, when I am trying to export the trained model to
SavedModel format, it fails. Here’s the full error trace.
Here’s the notebook for the training code. I use the following lines of code in order to export the trained model to
Where are you running this notebook, example, in Google Colab?
Hi @Karthik12. I am running it on Kaggle as a kernel.
Hi @sayakpaul, great to see you here
this notebook might help. It exports
Thank you, @valhalla. I will give it a check and update
Looks like it was a silly incompatibility. With the GPU version of the same notebook (hyperparameters changed slightly), I was able to export the model to
SavedModel. Does not look like a bug in
transformers. Might be stemming from some incompatible Op within TensorFlow itself.
Looking at the stack trace, I guess it’s because TPU strategy can only access GCS file system, which is why it’s throwing
UnimplementedError: File system scheme '[local]' not implemented.
Maybe pointing to
gs:// address should allow export on TPU.
Just a guess, not super familiar with TF.
To be frank, huggingface support for Tensorflow is bare minimal. Its mostly focus on pytorch.
Yeah definitely. But it’s possible to export a model locally that’s been trained in a TPU runtime.
@sayakpaul Its not possible.