Hi! Thanks for the response. With ignore_verifications=True
I get this new error:
Traceback (most recent call last):
File “stance_detection.py”, line 167, in run
dataset_de = load_dataset(‘strombergnlp/x-stance’, ‘de’, ignore_verifications=True)
File “/home/mkon/miniconda3/envs/stance/lib/python3.7/site-packages/datasets/load.py”, line 1736, in load_dataset
use_auth_token=use_auth_token,
File “/home/mkon/miniconda3/envs/stance/lib/python3.7/site-packages/datasets/builder.py”, line 614, in download_and_prepare
dl_manager=dl_manager, verify_infos=verify_infos, **download_and_prepare_kwargs
File “/home/mkon/miniconda3/envs/stance/lib/python3.7/site-packages/datasets/builder.py”, line 702, in _download_and_prepare
self._prepare_split(split_generator, **prepare_split_kwargs)
File “/home/mkon/miniconda3/envs/stance/lib/python3.7/site-packages/datasets/builder.py”, line 1167, in _prepare_split
writer.write_table(table)
File “/home/mkon/miniconda3/envs/stance/lib/python3.7/site-packages/datasets/arrow_writer.py”, line 523, in write_table
pa_table = table_cast(pa_table, self._schema)
File “/home/mkon/miniconda3/envs/stance/lib/python3.7/site-packages/datasets/table.py”, line 1895, in table_cast
return cast_table_to_schema(table, schema)
File “/home/mkon/miniconda3/envs/stance/lib/python3.7/site-packages/datasets/table.py”, line 1878, in cast_table_to_schema
arrays = [cast_array_to_feature(table[name], feature) for name, feature in features.items()]
File “/home/mkon/miniconda3/envs/stance/lib/python3.7/site-packages/datasets/table.py”, line 1878, in
arrays = [cast_array_to_feature(table[name], feature) for name, feature in features.items()]
File “/home/mkon/miniconda3/envs/stance/lib/python3.7/site-packages/datasets/table.py”, line 1673, in wrapper
return pa.chunked_array([func(chunk, *args, **kwargs) for chunk in array.chunks])
File “/home/mkon/miniconda3/envs/stance/lib/python3.7/site-packages/datasets/table.py”, line 1673, in
return pa.chunked_array([func(chunk, *args, **kwargs) for chunk in array.chunks])
File “/home/mkon/miniconda3/envs/stance/lib/python3.7/site-packages/datasets/table.py”, line 1843, in cast_array_to_feature
return array_cast(array, feature(), allow_number_to_str=allow_number_to_str)
File “/home/mkon/miniconda3/envs/stance/lib/python3.7/site-packages/datasets/table.py”, line 1675, in wrapper
return func(array, *args, **kwargs)
File “/home/mkon/miniconda3/envs/stance/lib/python3.7/site-packages/datasets/table.py”, line 1752, in array_cast
return array.cast(pa_type)
File “pyarrow/array.pxi”, line 915, in pyarrow.lib.Array.cast
File “/home/mkon/miniconda3/envs/stance/lib/python3.7/site-packages/pyarrow/compute.py”, line 376, in cast
return call_function(“cast”, [arr], options)
File “pyarrow/_compute.pyx”, line 542, in pyarrow._compute.call_function
File “pyarrow/_compute.pyx”, line 341, in pyarrow._compute.Function.call
File “pyarrow/error.pxi”, line 144, in pyarrow.lib.pyarrow_internal_check_status
File “pyarrow/error.pxi”, line 100, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: Failed to parse string: ‘AGAINST’ as a scalar of type int64
Here, AGAINST
is one of the two class labels for the dataloader, the other being FAVOR
. The dataloader passes the test suite so unsure why it crashes here.