Hi, for sequence to sequence task in tensorflow, I see that I can use TFT5 model. However, before sequence to sequence training, I need to perform masked language model pretraining on the encoder and initialise the weights of the encoder and decoder with the same weights that I obtain by masked language model pretraining. Is there a way to do this in the tensorflow version of T5?
I could do that with the EncoderDecoder, but it is only supported in PyTorch and not tensorflow.
Please help.