Some tasks like question generation requires multiple metrics (BLEU, METEOR, ROUGE).
It would be quite helpful if there is a function such as load_metric(['bleu', 'meteor', 'rouge'])
hey @ad26kr, thanks for the suggestion!
one complication i see is that some metrics like glue are paired a subtask:
metric = load_metric('glue', sub_task)
so it’s not clear what should happen if someone passes something like load_metric(["bleu", "glue"]) in your proposal. nevertheless, i invite you to open a feature request on the datasets library where it can be discussed in more detail: Issues · huggingface/datasets · GitHub
Do we still have this feature?
Please. Did you add multi-metric feature to huggingface or not yet?