Hey @cruiser 
Sadly, I think you have to do it manually – i.e. save your model (including the optimizer) using native TF functions, then push all resulting files to the hub.
We have some functionality for Functional/Sequential Keras models (here), but transformer models are Subclassed models.