I have a list of tokens and I want to understand the logic behind tokenizer.convert_tokens_to_string
but I can’t seem to find the source code for this function or debug my way to it.
Can you help?
I have a list of tokens and I want to understand the logic behind tokenizer.convert_tokens_to_string
but I can’t seem to find the source code for this function or debug my way to it.
Can you help?