Get UserWarning: "Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector" when run official example code run_mlm.py

System info are:
Platform: Linux-4.15.0-189-generic-x86_64-with-Ubuntu-18.04-bionic
Python 3.8.13
transformers 4.23.1
huggingface-hub 0.10.1
torch 1.12.1

I just run the official example scripts on transformers/examples/pytorch/language-modeling at eca77f4719531ecaabe9ec6b2dee6075a391d98a · huggingface/transformers · GitHub about bert mlm train.

python run_mlm.py
–model_name_or_path roberta-base
–train_file path_to_train_file
–validation_file path_to_validation_file
–per_device_train_batch_size 8
–per_device_eval_batch_size 8
–do_train
–do_eval
–output_dir ./tmp/test-mlm

and I get a warning info like:
python3.8/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.

I want to know is this a normal situation? Thanks
@sgugger

1 Like

Facing similar issue, were you able to figure out the cause?