Hugging Face Forums
UTF-16 for datasets?
🤗Datasets
mariosasko
June 19, 2023, 7:16pm
2
You can pass
encoding="utf-16"
to the
load_dataset
call.
2 Likes
Random utf-8 errors from dataset
show post in topic
Related topics
Topic
Replies
Views
Activity
Issues with non-ASCII symbols in Datasets Viewer
Site Feedback
1
1096
September 17, 2021
UniDecodeError: 'charmap' codec can't decode byte from Load_dataset
Beginners
0
40
December 5, 2024
Random utf-8 errors from dataset
Intermediate
10
3082
December 8, 2023
How to ensure that the escapes for the double quotes '\"' inside the 'user content' for the training datasets will not be removed?
🤗Datasets
0
130
April 11, 2024
Datasets.load_datasets fails
🤗Datasets
12
525
October 11, 2024