What version of torch are you using? <=1.13.1?
I am seeing a regression with 4.38.* and torch<=1.13.1.
dataloader_prefetch_factor was added to TrainingArguments 2 months ago with the default value None: Blaming transformers/src/transformers/training_args.py at e9476832942a19cf99354776ef112babc83c139a · huggingface/transformers · GitHub
But old versions of torch do not accept None and will raise an error if num_workers == 0 and prefetch_factor != 2: pytorch/torch/utils/data/dataloader.py at 49444c3e546bf240bed24a101e747422d1f8a0ee · pytorch/pytorch · GitHub