UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector when running trainer

Beginners
1

This is my code


training_args = TrainingArguments(
    learning_rate               = 3e-5,
    max_grad_norm = 1.0,
    #weight_decay                 = 0.01,
    num_train_epochs            = 3,
    per_device_train_batch_size = 32,
    per_device_eval_batch_size  = 1,
    logging_steps               = 300,
    output_dir                  = "./training_output",
    overwrite_output_dir        = True,
    seed =42,
    fp16=True,
    remove_unused_columns       = False
)

trainer = Trainer(
    model           = model,
    args            = training_args,
    train_dataset = train
)
trainer.args._n_gpu = 2

When I use trainer.train() the model run okay but at the end says “UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead uns”, do you know why or how to fix it?

2 Likes
2

@Mari22 did you find a fix for this? I seem to be running into the same issue.

1 Like