Hi, as the title said. I just don’t understand the use-cases of this context manager
with tokenizer.as_target_tokenizer():
I have read the doc though
THANKS
Hi, as the title said. I just don’t understand the use-cases of this context manager
with tokenizer.as_target_tokenizer():
I have read the doc though
THANKS
according to understand differences in tokenization · Issue #18455 · huggingface/transformers · GitHub
this context manager is deprecated so don’t worry