Validation Loss for VITMAE

🤗Transformers
1

I’ve started using VITMAE, adapting code from the script run_mae.py.

I’m unable to get any validation loss - in spite of setting evaluation_strategy=epoch and providing an evaluation set to the trainer. I get Validation Loss: “No log” for all epochs. Specifying compute_metrics() results in an exception related to tensors

I’m wondering if this is because the trainer expects labels for evaluation, and the model doesn’t provide them during pre-training. Is there a workaround other than a custom training loop?

Thanks!

2

When you construct your TrainingArguments, you need to add the field as:

training_args = TrainingArguments( .... label_names=["pixel_values"], .... )

I’m also assuming you are setting eval_dataset=<your_eval_dataset> in your Trainer, right?

During validation some “label” is required to compute the metric. In this case, the input is the label, so when you specify this it should work.

1 Like