Hi, I want to train a sentence pair classification model where 2 input sentences are from different languages. I am confused about the tokenization step. Should I use 2 different tokenizers for the 2 sentences and then concatenate?
Hi, I want to train a sentence pair classification model where 2 input sentences are from different languages. I am confused about the tokenization step. Should I use 2 different tokenizers for the 2 sentences and then concatenate?