Can't load exist dataset for evaluation

šŸ¤—Datasets
1

Hi, I used dair-ai/emotion to fine-tune the bert-base-cased model to use it for text-classification task.

The dataset is loaded when I load it in this script, so there is no problem up to here:

from datasets import load_dataset

dataset = load_dataset("dair-ai/emotion")

But in the evaluation script here:


import numpy as np
import evaluate

predictions = trainer.predict(eval_dataset)

preds = np.argmax(predictions.predictions, axis=-1)

metric = evaluate.load("dair-ai/emotion")

metric.compute(predictions=preds, references=predictions.label_ids)

I got: FileNotFoundError: Couldnā€™t find a module script at /content/dair-ai/emotion/emotion.py. Module ā€˜dair-ai/emotionā€™ doesnā€™t exist on the Hugging Face Hub either.

I tried to find out the reason but did not find the solution.

2

Hellllo :))

3

Does the issue I recently filed (DataFilesNotFoundError for datasets in the open-llm-leaderboard) sound like what youā€™re experiencing? There was no resolution, but perhaps youā€™re at least not alone

4

No, it is not the same. All that matters is that the dataset already exists on the Hub and I used it in the data preparation phase, but at the evaluation phase there is an error saying that the dataset does not exist on the Hub.

I think the problem is from the evaluate lib,
And I have already opened an Issue on GitHub, but on evaluate repo.

issue link

1 Like