Loading the pile

🤗Datasets
1

I’m not able to load the pile dataset. Here is the code I’m using

dataset_name = "EleutherAI/the_pile"
dataset = load_dataset(dataset_name, subsets = ['hacker_news', 'enron_emails'])

Here is the error I’m getting

TypeError: Couldn't cast array of type
struct<file: string, id: string>
to
{'id': Value(dtype='string', id=None)}

Any ideas why?

2

Hi! We merged a fix a couple of days ago, so it should work now.

3

still doesnt work- did you see my comment on the dataset on hf?