Hello, there. I downloaded a dataset in Hub and saved it to a local folder. However I could not reload it. Here is my code from datasets import load_dataset raw_datasets = load_dataset("roneneldan/TinyStories") raw_datasets.save_to_disk("Tiny_Stories") raw_datasets = load_dataset('text', data_dir =…

Hi! save_to_disk doesn’t save a dataset as a text file (writes Arrow and JSON metadata files), hence the error. Instead, use load_from_disk("Tiny_Stories") to load the dataset.

'utf-8' codec can't decode byte 0xff in position 0: invalid start byte

Beginners

Toflamus August 23, 2023, 1:55pm 4

Thank you very much!!! Problem solved! Your answer really helps me a lot!

Datasets.load_datasets fails

Topic		Replies	Views
Datasets.load_datasets fails 🤗Datasets	12	919	October 11, 2024
Random utf-8 errors from dataset Intermediate	10	3738	December 8, 2023
'utf-8' codec can't decode byte 0xff in position 0: invalid start byte (dataset) Beginners	0	402	May 19, 2024
Save and load datasets 🤗Datasets	2	40479	August 16, 2021
Load_dataset did not load the text file? Beginners	0	653	June 7, 2021

'utf-8' codec can't decode byte 0xff in position 0: invalid start byte

Related topics