How to predict in Tensorflow

🤗Transformers
#1

Hi,
I have just finetuned RoBERTa for a classification problem, trained and stored the model. I have used native Tensorflow throughout, but I can’t find any examples anywhere related to finally predicting in TF. The only ones I found were all for PyTorch.

Can someone please guide me to some snippet of code for prediction? The official example has no such code…

EDIT:- I tried using model.predict() method that is used normally and even the PyTorch one which reports missing config. I am really confused as to how we are supposed to serve predictions. IMO the docs need updating

#2

Hi @Neel-Gupta,

The steps are usually as follows

You load your trained model similar to this

loaded_model = TFDistilBertForSequenceClassification.from_pretrained("./sentiment")

Then you usually have to run your text through the preprocessing in the same way you did for your training. Usually some kind of tokenizing involved

predict_input = tokenizer.encode(test_sentence,
                                 truncation=True,
                                 padding=True,
                                 return_tensors="tf")

And then you finally can predict based on your loaded model and the preprocessed text

tf_output = loaded_model.predict(predict_input)[0]

Disclaimer, I am the author of this article, but It covers the process from data to deployment.
https://blog.doit-intl.com/performing-surprisingly-easy-sentiment-analysis-on-google-cloud-platform-fc26b2e2b4b