Given this notebook from huggingface
If I want to use other dataset , like this one, Streamlit
the dataset has list kind of field for ‘context’,
please check is my change correct to make it a string (I am getting error without this)
for i, example in enumerate(datasets[“train”]):
if len(tokenizer(example[“question”], str(example[“context”][“contexts”]))[“input_ids”]) > 384:
break
example = datasets[“train”][i]
initially, it was
for i, example in enumerate(datasets[“train”]):
if len(tokenizer(example[“question”], example[“context”])[“input_ids”]) > 384:
break
example = datasets[“train”][i]