Huggingface Trainer eval while training

🤗Transformers
1

when I ran this script run_mlm.py, I set do eval every 2000 steps, but after first 2000 steps training and start to eval，I got this problem：

    train_result = trainer.train(resume_from_checkpoint=checkpoint)
  File "/home/yzhang/anaconda3/envs/ke/lib/python3.8/site-packages/transformers/trainer.py", line 1501, in train
    return inner_training_loop(
  File "/home/yzhang/anaconda3/envs/ke/lib/python3.8/site-packages/transformers/trainer.py", line 1826, in _inner_training_loop
    self._maybe_log_save_evaluate(tr_loss, model, trial, epoch, ignore_keys_for_eval)
  File "/home/yzhang/anaconda3/envs/ke/lib/python3.8/site-packages/transformers/trainer.py", line 2089, in _maybe_log_save_evaluate
    metrics = self.evaluate(ignore_keys=ignore_keys_for_eval)
  File "/home/yzhang/anaconda3/envs/ke/lib/python3.8/site-packages/transformers/trainer.py", line 2796, in evaluate
    output = eval_loop(
  File "/home/yzhang/anaconda3/envs/ke/lib/python3.8/site-packages/transformers/trainer.py", line 2974, in evaluation_loop
    loss, logits, labels = self.prediction_step(model, inputs, prediction_loss_only, ignore_keys=ignore_keys)
  File "/home/yzhang/anaconda3/envs/ke/lib/python3.8/site-packages/transformers/trainer.py", line 3239, in prediction_step
    logits = nested_detach(logits)
  File "/home/yzhang/anaconda3/envs/ke/lib/python3.8/site-packages/transformers/trainer_pt_utils.py", line 165, in nested_detach
    return type(tensors)(nested_detach(t) for t in tensors)
  File "/home/yzhang/anaconda3/envs/ke/lib/python3.8/site-packages/transformers/trainer_pt_utils.py", line 165, in <genexpr>
    return type(tensors)(nested_detach(t) for t in tensors)
  File "/home/yzhang/anaconda3/envs/ke/lib/python3.8/site-packages/transformers/trainer_pt_utils.py", line 168, in nested_detach
    return tensors.detach()
AttributeError: 'float' object has no attribute 'detach'

This is a error in eval. My eval_dataset has the same structure with train_dataset.
How should I do？
Thanks a lot.

2

Could you provide us the exact command you ran to launch this script?