'datasets.iterable_dataset.IterableDataset' to 'datasets.dataset_dict.DatasetDict'

🤗Datasets
1

how can i convert <class ‘datasets.iterable_dataset.IterableDataset’> into <class ‘datasets.dataset_dict.DatasetDict’>

1 Like
2

This should work:

from datasets import Dataset, DatasetDict

ds = Dataset.from_generator(lambda: (yield from iterable_ds), features=iterable_ds.features)
dd = DatasetDict({"train": ds})
3

MY data set type is “<class ‘datasets.dataset_dict.IterableDatasetDict’>” still i got his error “AttributeError: ‘IterableDatasetDict’ object has no attribute ‘features’” when i ran your code "from datasets import Dataset, DatasetDict

ds = Dataset.from_generator(lambda: (yield from train_dataset), features=train_dataset.features)
dd = DatasetDict({“train”: ds})"

4

Try this then:

dd = DatasetDict()
for ds_name, iterable_ds in iterable_dd.items():
    ds = Dataset.from_generator(lambda: (yield from iterable_ds), features=iterable_ds.features)
    dd[ds_name] = ds