I am trying to load a dataset from the openx datasets using a command like this: dataset = datasets.load_dataset( “jxu124/OpenX-Embodiment”, “berkeley_gnm_cory_hall”, streaming=False, split=“train”, cache_dir=ds_root, trust_remote_code=True, ) But it’s failing on any argument configurations…

Datasets.load_datasets fails

lhoestq October 7, 2024, 12:54pm 8

We’re using PyArrow to load the data, maybe we can see with them how to improve it. Do you have a reproducible example ?

1 Like

Topic		Replies	Views
'utf-8' codec can't decode byte 0xff in position 0: invalid start byte Beginners	3	12286	August 23, 2023
Random utf-8 errors from dataset Intermediate	10	3822	December 8, 2023
UniDecodeError: 'charmap' codec can't decode byte from Load_dataset Beginners	0	70	December 5, 2024
'utf-8' codec can't decode byte 0xff in position 0: invalid start byte (dataset) Beginners	0	420	May 19, 2024
UTF-16 for datasets? 🤗Datasets	4	1533	June 21, 2023