How to apply DataCollator to a dataset for evaluation?

i have a test dataset with the following structure:

tokenized_data
=> Dataset({
    features: ['id', 'tokens', 'ner_tags', 'input_ids', 'token_type_ids', 'attention_mask', 'labels'],
    num_rows: 2672
})

I am trying to use the Evaluator class like so:

task_evaluator = evaluate.evaluator('token-classification')
eval_results = task_evaluator.compute(model_or_pipeline=model,
                                      tokenizer=tokenizer,
                                      data=tokenized_data,
                                      metric='seqeval')

However, data is not truncated yet and therefore i get an error:

RuntimeError: The size of tensor a (951) must match the size of tensor b (512) at non-singleton dimension 1

Is there any way to pass a DataCollator object to the evaluator?