Custom SQuAD2.0 dataset gives an error when using run_qa.py script

🤗Transformers
1

Hello,

I am trying to follow the PyTorch Question Answering example. However, when running the run_qa.py script using my own (Dutch machine-translated) SQuAD train and test files (JSON), I get the following error: pyarrow.lib.ArrowInvalid: cannot mix list and non-list, non-null values.

I use the following hyperparameters:

python run_qa.py \
--model_name_or_path GroNLP/bert-base-dutch-cased \
--version_2_with_negative \
--do_train \
--do_eval \
--train_file "C:\Users\myname\data\squad\nl_squad_train_clean.json" \
--test_file "C:\Users\myname\data\squad\nl_squad_dev_clean.json" \
--per_device_train_batch_size 12 \
--num_train_epochs 2 \
--max_seq_length 384 \
--doc_stride 128 \
--save_steps=800 \
--output_dir ../output

When replacing the train and test by --dataset_name squad it works fine. What could be the problem with my own SQuAD files?

Thanks in advance! Cheers!

2

hey @julifelipe, my guess is that you have your squad data in the conventional json format, while the run_qa.py expects the examples to be line-delimited json

you can find a simple function to do the conversion here: Question answering bot: fine-tuning with custom dataset - #2 by lewtun

1 Like
3

Hey @lewtun, thanks for pointing me in the right direction! Only now I get the following error: TypeError: list indices must be integers or slices, not str. I get the gist of this error but I don’t understand why it shows up here, seeing as I’m not doing anything crazy.

TypeError                                 Traceback (most recent call last)
/var/folders/yb/0cqx_6j50191l1m_p9p30gbw0000gn/T/ipykernel_13798/3502567035.py in <module>
     16                 question = qa["question"]
     17                 idx = qa["id"]
---> 18                 answers["text"] = [a["text"] for a in qa["answers"]]
     19                 answers["answer_start"] = [a["answer_start"] for a in qa["answers"]]
     20                 f.write(

/var/folders/yb/0cqx_6j50191l1m_p9p30gbw0000gn/T/ipykernel_13798/3502567035.py in <listcomp>(.0)
     16                 question = qa["question"]
     17                 idx = qa["id"]
---> 18                 answers["text"] = [a["text"] for a in qa["answers"]]
     19                 answers["answer_start"] = [a["answer_start"] for a in qa["answers"]]
     20                 f.write(

TypeError: list indices must be integers or slices, not str
4

hey @julifelipe my suggestion would be to compare how your schema compares against that in the squad dataset and see what might be going wrong in the fields.

e.g.

from datasets import load_dataset

squad = load_dataset("squad", split="train")
# compare your examples against this
squad.to_json("squad.jsonl")
# first example
# {"id":"5733be284776f41900661182","title":"University_of_Notre_Dame","context":"Architecturally, the school has a Catholic character. Atop the Main Building's gold dome is a golden statue of the Virgin Mary. Immediately in front of the Main Building and facing it, is a copper statue of Christ with arms upraised with the legend \"Venite Ad Me Omnes\". Next to the Main Building is the Basilica of the Sacred Heart. Immediately behind the basilica is the Grotto, a Marian place of prayer and reflection. It is a replica of the grotto at Lourdes, France where the Virgin Mary reputedly appeared to Saint Bernadette Soubirous in 1858. At the end of the main drive (and in a direct line that connects through 3 statues and the Gold Dome), is a simple, modern stone statue of Mary.","question":"To whom did the Virgin Mary allegedly appear in 1858 in Lourdes France?","answers":{"text":["Saint Bernadette Soubirous"],"answer_start":[515]}}