Hello all, first off, thank you so much for developing huggingface transformers and making NLP available to all. I am attempting to finetune a BART model for text-simplification, and I want to use the SARI metric as the metric for the trainer. However, since SARI expects sources in addition to predictions and references, I am not sure how to implement SARI in the compute_metrics() function for the trainer. I currently understand that the EvalPrediction object that is expected for the compute_metrics() function only has the predictions and references. How may I add sources as well?
2 Likes
Did you get the solution for this?