Passing additional tensors into metric evaluation function

I was wondering if it is possible to pass additional labels into metric evaluation function. In more details, I want to calculate metrics using not all predicted tokens but only a small subset, and to do this I need to pass a tensor of indices of that tokens into the metric evaluation function. But I don’t need this tensor for anything else, e.g. for calculating the loss.
I tried to add this tensor into evaluation dataset, using label_names and remove_unused_columns = False options. But in this case the model (T5 encoder-decoder) sees unrecognized keywords and fails. Default remove_unused_columns = True just deletes the tensor from the batch.
I am using :hugs: Transfomers version 4.7.0

Any help is much appreciated! Thanks.

If you leave it in your dataset, you will have an error since your model can’t consume it. You will need to write a manual evaluation loop for this use case.

Got it. Thank you for the answer.