How is compute_metrics working internally?

Beginners
1

Hi everyone, I am following this blog post Fine-Tune XLSR-Wav2Vec2 for low-resource ASR with 🤗 Transformers on fine-tuning an ASR model, and there is something I don’t understand about the compute_metrics function.

In the notebook, we want to compute the Word Error Rate for the validation set, every eval_steps steps. What is the input of this function? Does it take one batch at a time, or the whole validation dataset? If it takes one batch at a time, how is the final WER that is diplayed calculated? Is it the mean of all the WERs of all the batches?

Pinging the author of this blog post as well @patrickvonplaten . I’d appreciate any insight you guys can give me.

Thanks in advance.

2

The compute_metrics function takes the predictions and labels over the whole evaluation dataset and computes the metrics from them.

1 Like
3

Thank you.