i have a test dataset with the following structure:
tokenized_data
=> Dataset({
features: ['id', 'tokens', 'ner_tags', 'input_ids', 'token_type_ids', 'attention_mask', 'labels'],
num_rows: 2672
})
I am trying to use the Evaluator class like so:
task_evaluator = evaluate.evaluator('token-classification')
eval_results = task_evaluator.compute(model_or_pipeline=model,
tokenizer=tokenizer,
data=tokenized_data,
metric='seqeval')
However, data is not truncated yet and therefore i get an error:
RuntimeError: The size of tensor a (951) must match the size of tensor b (512) at non-singleton dimension 1
Is there any way to pass a DataCollator object to the evaluator?