Transformers and TensorFlow Extended (TFX)

Interested in the TFX support of Hugginface Transformers.
Referring to this blog post

The library has seen super-fast growth in PyTorch and has recently been ported to TensorFlow 2.0, offering an API that now works with Keras’ fit API, TensorFlow Extended, and TPUs :clap:.

To be precise for the preprocessing part we are using TensorFlow Transform as part of the TFX pipeline.

  • Are there any reference implementations or experience with using Hugginface Transformers as part of the TensorFlow Transform (TFX Transform Component)?
  • And even more important the possibility to integrate the preprocessing steps as part of the model graph?
  • To integrate the transformation process the complete implementation needs to be implemented with TensorFlow Operations. Is this supported? I Assume if there is support for TPUs it is the case, but I could not find any documentation.

Looking forward if someone has experimented with it already.
Happy to join a joint force to dig deeper into these possibilities.

1 Like