Hello, there. I downloaded a dataset in Hub and saved it to a local folder. However I could not reload it. Here is my code from datasets import load_dataset raw_datasets = load_dataset("roneneldan/TinyStories") raw_datasets.save_to_disk("Tiny_Stories") raw_datasets = load_dataset('text', data_dir =…

Hi! save_to_disk doesn’t save a dataset as a text file (writes Arrow and JSON metadata files), hence the error. Instead, use load_from_disk("Tiny_Stories") to load the dataset.

'utf-8' codec can't decode byte 0xff in position 0: invalid start byte

Beginners

Toflamus August 23, 2023, 1:55pm 4

Thank you very much!!! Problem solved! Your answer really helps me a lot!

Datasets.load_datasets fails

Topic		Replies	Views
'utf-8' codec can't decode byte 0xff in position 0: invalid start byte (dataset) Beginners	0	229	May 19, 2024
Random utf-8 errors from dataset Intermediate	10	2646	December 8, 2023
Datasets.load_datasets fails 🤗Datasets	12	71	October 11, 2024
Problem reading my own dataset 🤗Datasets	0	149	May 26, 2024
UnicodeDecodeError when loading Mulit Lingual text file 🤗Datasets	1	2258	April 7, 2022

'utf-8' codec can't decode byte 0xff in position 0: invalid start byte

Related Topics