When trying to use EarlyStopping for Seq2SeqTrainer, e.g. patience was set to 1 and threshold 1.0:
training_args = Seq2SeqTrainingArguments( output_dir='./', num_train_epochs=3, per_device_train_batch_size=4, per_device_eval_batch_size=4, logging_steps=1, save_steps=5, eval_steps=1, max_steps=10, evaluation_strategy="steps", predict_with_generate=True, report_to=None, metric_for_best_model="chr_f_score", load_best_model_at_end=True ) early_stop = EarlyStoppingCallback(2, 1.0) trainer = Seq2SeqTrainer( model=model, args=training_args, train_dataset=valid_data.with_format("torch"), eval_dataset=test_data.with_format("torch"), compute_metrics=compute_metrics, callbacks=[early_stop] ) trainer.train()
The model continues training until
max_steps instead of stopping after the stopping criteria is met.
I’m not sure if this is a bug or maybe some argument is missing in when I use Seq2SeqTrainer, a working code to replicate the issue can be found on Huggingface EarlyStopping Callbacks | Kaggle
max_steps, if we do some probing, somehow the
early_stopping_patience_counter has been reached but the training didn’t stop
>>> early_stop.early_stopping_patience_counter 2