I’m trying to load the common voice 17 HU dataset, but I always get an error when it gets to generating examples, the error:
Extracting data files: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 6/6 [00:00<00:00, 114.46it/s]
Reading metadata…: 37140it [00:00, 133407.71it/s]es/s]
Generating train split: 0 examples [00:00, ? examples/s]Traceback (most recent call last):
File “C:\Users\tothg\AppData\Local\Programs\Python\Python312\Lib\site-packages\datasets\builder.py”, line 1627, in _prepare_split_single
example = self.info.features.encode_example(record) if self.info.features is not None else record
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “C:\Users\tothg\AppData\Local\Programs\Python\Python312\Lib\site-packages\datasets\features\features.py”, line 1813, in encode_example
return encode_nested_example(self, example)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “C:\Users\tothg\AppData\Local\Programs\Python\Python312\Lib\site-packages\datasets\features\features.py”, line 1212, in encode_nested_example
{
File “C:\Users\tothg\AppData\Local\Programs\Python\Python312\Lib\site-packages\datasets\utils\py_utils.py”, line 302, in zip_dict
yield key, tuple(d[key] for d in dicts)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “C:\Users\tothg\AppData\Local\Programs\Python\Python312\Lib\site-packages\datasets\utils\py_utils.py”, line 302, in
yield key, tuple(d[key] for d in dicts)
~^^^^^
KeyError: ‘sentence_id’
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File “E:\torch-directml\dl_dataset.py”, line 3, in
train = load_dataset(“mozilla-foundation/common_voice_17_0”, “hu”, use_auth_token=“”)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “C:\Users\tothg\AppData\Local\Programs\Python\Python312\Lib\site-packages\datasets\load.py”, line 1791, in load_dataset
builder_instance.download_and_prepare(
File “C:\Users\tothg\AppData\Local\Programs\Python\Python312\Lib\site-packages\datasets\builder.py”, line 891, in download_and_prepare
self._download_and_prepare(
File “C:\Users\tothg\AppData\Local\Programs\Python\Python312\Lib\site-packages\datasets\builder.py”, line 1651, in _download_and_prepare
super()._download_and_prepare(
File “C:\Users\tothg\AppData\Local\Programs\Python\Python312\Lib\site-packages\datasets\builder.py”, line 986, in _download_and_prepare
self._prepare_split(split_generator, **prepare_split_kwargs)
File “C:\Users\tothg\AppData\Local\Programs\Python\Python312\Lib\site-packages\datasets\builder.py”, line 1490, in _prepare_split
for job_id, done, content in self._prepare_split_single(
File “C:\Users\tothg\AppData\Local\Programs\Python\Python312\Lib\site-packages\datasets\builder.py”, line 1646, in _prepare_split_single
raise DatasetGenerationError(“An error occurred while generating the dataset”) from e
datasets.builder.DatasetGenerationError: An error occurred while generating the dataset
The code is just:
from datasets import load_dataset
train = load_dataset(“mozilla-foundation/common_voice_17_0”, “hu”, use_auth_token=“”)