Connection error with bleu metric in 🤗 Evaluate


I tried to use the bleu score provided by Evaluate instead of Datasets as it doesn’t need the data to be tokenized. It worked well in Colab; but when I was testing it locally (Linux), I got the following error:

ConnectionError: Couldn't reach (ReadTimeout(ReadTimeoutError("HTTPSConnectionPool(host='', port=443): Read timed out. (read timeout=100)")))

I checked my local hosts in /etc/hosts, and it did have the… in it, so I assume that it should be possible for the metric to be downloaded. And when I clicked on the URL, I was redirected to…. I don’t quite see why the site the script tried to reach is not, maybe that’s the reason to this…

I just ran the following code on Colab and can’t reproduce the issue.

import evaluate

As far as I can tell the github link gets redirected to so maybe the first call failed? Does the issue persist?

I did try to run the code in Colab and it was fine. Guess the server that I ran my code on wasn’t configured properly…

I have changed to use load_metric() in :hugs: datasets.

So it works with load_metric in datasets but not with load in evaluate? The main difference between the two is that load will now get the metric from the Hugging Face Hub whereas load_metric will get it from GitHub. Maybe your server blocked that address?

I think this is the case. Since I’ve been using load_metric() there has been no problem. I did communicate with the superuser of the server and the websites are not properly configured.

Thank you for your reply! I think this thread can be viewed as solved now.