Connection error with bleu metric in 🤗 Evaluate

Hi,

I tried to use the bleu score provided by Evaluate instead of Datasets as it doesn’t need the data to be tokenized. It worked well in Colab; but when I was testing it locally (Linux), I got the following error:

ConnectionError: Couldn't reach https://github.com/tensorflow/nmt/raw/master/nmt/scripts/bleu.py (ReadTimeout(ReadTimeoutError("HTTPSConnectionPool(host='github.com', port=443): Read timed out. (read timeout=100)")))

I checked my local hosts in /etc/hosts, and it did have the raw.githubusercontent.com/… in it, so I assume that it should be possible for the metric to be downloaded. And when I clicked on the URL, I was redirected to raw.githubusercontent.com/…. I don’t quite see why the site the script tried to reach is github.com not raw.githubusercontent.com, maybe that’s the reason to this…

I just ran the following code on Colab and can’t reproduce the issue.

import evaluate
evaluate.load("bleu")

As far as I can tell the github link gets redirected to raw.githubusercontent.com so maybe the first call failed? Does the issue persist?

I did try to run the code in Colab and it was fine. Guess the server that I ran my code on wasn’t configured properly…

I have changed to use load_metric() in :hugs: datasets.

So it works with load_metric in datasets but not with load in evaluate? The main difference between the two is that load will now get the metric from the Hugging Face Hub whereas load_metric will get it from GitHub. Maybe your server blocked that address?

I think this is the case. Since I’ve been using load_metric() there has been no problem. I did communicate with the superuser of the server and the websites are not properly configured.

Thank you for your reply! I think this thread can be viewed as solved now.