Hi Christoph! Thanks for the question, and sorry about the delay. In the guide, we didn’t use the evaluation in the Trainer, because accuracy usually isn’t used for object detection. Here, we evaluated separately on standard COCO metrics. If you want to measure accuracy during training, you’ll need to check how the labels are batched in collate_fn(batch), (e.g. use batch[“labels”] = [dict(label) for label in labels] instead). You may also need to modify your compute_metrics function.
If this doesn’t work, @ybelkada may be able to help with the evaluation.