Any examples of how to grpc call a tensorflow_model_server running a T5 model?
What I did so far is:
- Saving a model in torch format using: model.save_pretrained(“local-pt-checkpoint”)
- Convert the saved model to ONNX using: python -m transformers.onnx --model=local-pt-checkpoint --opset=13 onnx/
- Convert the ONNX to tf using: onnx-tf convert -i onnx/model.onnx -o tf_export
- Run the model server using: tensorflow_model_server --port=9000 --model_name=my_model --model_base_path=./t5-base/tf_export/.
Now I just need to find some example code of how to call the model server.