Evaluate.load() does not working but datasets.load_metric() does

🤗Evaluate
#1

Hi. I have a custom metric (BLEURT in this case) which I am correctly loading using datasets.load_metric("metrics/bleurt"). As I saw the warning of deprecation, I tried switching to evaluate.load(). It should be pretty straight forward! but it wasn’t. With evaluate.load() I get the following error:
TypeError: 'NoneType' object is not callable
Are there new specifications the metrics should meet to be properly loaded using evaluate?

#2

Hi ! BLEURT works fine on my side using

import evaluate
bleurt = evaluate.load("bleurt")
bleurt.compute(predictions=["hello there"], references=["hey there"])

can you share your full stack trace and the version of evaluate and datasets you’re using ?

#3

It was an error on my part. I have a custom blurt because I want a bleurt score per pair or reference and prediction. The problem was that I forgot to change the super class from datasets.Metric to evaluate.Metric. Sorry for the inconvenience.