Loading list as dataset

Beginners
1

I understand typically I should save a list of data (e.g., list = [{"text": "...", "label": 1}, {"text": '...', "label": 1},...]) as a json file and use load_dataset("json", data_files="/path/data.json") to load as dataset. However in this use case, this list is generated on the fly, so I would like to load a list directly as a dataset. I used load_dataset(list) but got the following error:

TypeError: expected str, bytes or os.PathLike object, not list

Is there some way to allow me to directly load list as dataset?

2

Hi! You can use Dataset.from_list(list) to create a Dataset from a Python list.

PS: Dataset.from_list was added to datasets in version 2.5.0, so if your installation is older than that, update it with pip install -U datasets.

1 Like
3

worked perfectly. Thanks!