Is there a way to acquire sp_model variable from T5Tokenizer? Related code can be found at
When we initiate tokenizer with ex. “t5-base”, from_pretrained function only returns fast version which does not have sp_model variable.
My aim is to initiate text.SentencepieceTokenizer from sp_model. If there is a way, this will help converting t5 tokenizer easily into tensorflow.
Or if there is a alternative way, I’ll be glad to learn
Thanks all in advance,