ValueError: The batch received was empty, your model won't be able to train on it. Double-check that your training dataset contains keys expected by the model: args,kwargs,label_ids,label

I am encountering the same error when fine tuning pre trained model when splitting processing between 2 gpus using DataParallel.
Have you found a solution?