T5 Inference using tensorflow_model_server (with grpc)

Any examples of how to grpc call a tensorflow_model_server running a T5 model?

What I did so far is:

  1. Saving a model in torch format using: model.save_pretrained(“local-pt-checkpoint”)
  2. Convert the saved model to ONNX using: python -m transformers.onnx --model=local-pt-checkpoint --opset=13 onnx/
  3. Convert the ONNX to tf using: onnx-tf convert -i onnx/model.onnx -o tf_export
  4. Run the model server using: tensorflow_model_server --port=9000 --model_name=my_model --model_base_path=./t5-base/tf_export/.

Now I just need to find some example code of how to call the model server.