How does trainer handle lists with None items?

Beginners
1

I am working through SquAD 2.0 QA with Bert and I am trying to figure out how to deal with data that contains None in it with PyTorch.

For QA tasks, for validation/test data only, we have data contained under the key: 'offset_mapping' and some of its data looks like:

# [[None, None, None, None, None, None, None, None, None, (0, 3), (4, 10), (10, 11), (12, 13), (13, 19), (19, 20), (21, 23), (23, 25), (25, 28), (28, 30), (30, 3]]

In this guide: https://colab.research.google.com/github/huggingface/notebooks/blob/master/examples/question_answering.ipynb#scrollTo=s3cuvZsNleG2&line=4&uniqifier=1, data that looks like above is successfully fed into trainer.predict() as well as into torch data loaders. However, torch data loaders will not accept tensors with None in them. How is huggingface handling and getting around these None values?

I am having a hard time figuring that out from the source code here: transformers.trainer — transformers 4.1.1 documentation

2

The Trainer automatically ignores columns that do not match model arguments. In this case, “offset_mapping” is not an argument of the QA model, so it’s ignored inside the Trainer. That’s why it’s not a problem if it has None items.