I’m able to successfully train and save my tokenizer but then i cant reload it.
tokenizer.save(tokenizer_save_path+"tokenizer.json") #works
newTokenizer = Tokenizer.from_file(tokenizer_save_path+"tokenizer.json") #breaks
I always get this error:
Exception: data did not match any variant of untagged enum ModelWrapper at line 3258 column 3