Unable to resolve any data file after loading once

🤗Datasets
1

when I rerun my program, it occurs this error
" Unable to resolve any data file that matches ‘[’* train ‘]’ at /data2/whr/lzy/open_domain_data/retrieval/wiki_dpr with any supported extension [‘csv’, ‘tsv’, ‘json’, ‘jsonl’, ‘parquet’, ‘txt’, ‘zip’]", so how could i deal with this problem?
thx.
And below is my code .
image

2

Hi ! Just copy-pasting the answer I posted on GitHub at Unable to resolve any data file after loading once · Issue #3431 · huggingface/datasets · GitHub, in case anyone has a similar issue:

Hi ! load_dataset accepts as input either a local dataset directory or a dataset name from the Hugging Face Hub.

So here you are getting this error the second time because it tries to load the local wiki_dpr directory, instead of wiki_dpr from the Hub. It doesn’t work since it’s a cache directory, not a dataset directory in itself.

To fix that you can use another cache directory like cache_dir="/data2/whr/lzy/open_domain_data/retrieval/cache"

1 Like