How to write my own metrics if it is not in datasets.metrics

🤗Datasets
1

I would like to use a custom metric, probably mean squared error. It is in sklearn.metrics but not in datasets.metrics – what is the quickest way to do this if it is not available in datasets.metrics?

2

You could have a look at implementation of existing metrics available here on datasets repo. You can even use one of the simpler one like accuracy or f1 as base and then modify it for your case.

Instructions to load a custom metric are available on the documentation page.

3

Hi ! It would be nice to have the MSE metric in :hugs: Datasets.

If you are interested in contributing, feel free to open a PR on GitHub to add this metric to the list of supported metrics in this folder : datasets/metrics at master · huggingface/datasets · GitHub

4

metrics are now deprecated in datasets - please check the evaluate library from now on !