Input of compute_metrics in ASR model

Beginners
1

What should be the input of the function below?
Is it the model or a pass forward of the model?

def compute_metrics(pred):
    pred_logits = pred.predictions
    pred_ids = np.argmax(pred_logits, axis=-1)

    pred.label_ids[pred.label_ids == -100] = processor.tokenizer.pad_token_id

    pred_str = processor.batch_decode(pred_ids)
    # we do not want to group tokens when computing the metrics
    label_str = processor.batch_decode(pred.label_ids, group_tokens=False)

    wer = wer_metric.compute(predictions=pred_str, references=label_str)

    return {"wer": wer}

Also, I have one question about how the Trainer class works…
I mean, it encloses everything, but does it update the weights of the model automatically or it creates another instance for model?

2

The input is a namedtuple of type EvalPrediction. It should have one key predictions for the predictions (will have the structure as the output of your model, so one tensor if your model outputted one tensor, a tuple of two tensors if that’s what your model returns, et.) and one key label_ids that will contain all the labels.

The Trainer does update the weights of the model, otherwise, the model would not be… well… training :wink:

3

I didn’t get the part of trainer not updating the weights of the model… because training would be about updating the weights on the hidden layers after the transformers and before the output… so what exactly does it do? Also, after using it, I cannot use the model to predict results anymore :cry: