XLM classification non pre trained language


I am currently working on a Classification problem using xlm-mlm-17-1280 where we want to compare the results on a language the model is not pre-trained on. However I am running into issues with the tokenisation as the gold label dataset for the non pre-trained dataset follows a word by word labeling (A sentence like ‘He isn’t running fast’ is labeled by each word ‘He, isn’t running fast’ ). The XLM does not tokenize to the same, what would be a good way to go about this?