Hello everyone! I want to change a truncation order from left to right in the PreTrainedTokenizerBase class. So, I found that the “padding + truncating” process lives inside the _batch_encode_plus function.
And now I am stuck because this function should raise NotImplementedError. But somehow it works and completes a tokenization process… And I can`t find how to go “deeper” inside this function and see what is going further. Can someone please tell me what is going on inside this function and how to get there? Many thanks!