How to load a huggingface amnesty/qa dataset from local path?

Beginners
1

I have downloaded dataset in local directly from github:

brew install git-lfs
git lfs instal
git clone https://huggingface.co/datasets/explodinggradients/amnesty_qa

It has following directory structure:

image
image466Ă—456 33.7 KB

I want to load english.json using hugging face load_dataset.

The json format is like this:

{
"question": [..,..,],
"answer": [..,...,],
"contexts": [
        [..], [].. ],
}

from datasets import load_dataset
dataset = load_dataset(
    "explodinggradients/amnesty_qa",
    "english_v3",
    trust_remote_code=True
)

@ vpkprasanna

1 Like
2

As long as the path is correct, it seems that no special operation is required, but you may be fine without specifying trust_remote_code. Because it is not a remote code after downloading…

3

Not working! Does the code load_dataset runs amnesty_qa.py file to fetch english_v3 data?

1 Like