Can do early stopping?

I am using to fine tune a pretrained model at Huggingface. I want fine tuning to stop when eval loss stops decreasing. Is it possible to get early stop behavior with run_clm? Thanks.

Hey, you can tweak run_clm a bit , first import these

from transformers import Trainer, TrainingArguments, EarlyStoppingCallback, IntervalStrategy

and add these in TrainingArguments

training_args = TrainingArguments(...)

  1. Use load_best_model_at_end = True (EarlyStoppingCallback() requires this to be True).
  2. evaluation_strategy = 'steps'
  3. eval_steps = N (evaluate the loss after N steps).

After use

trainer = Trainer(
    callbacks = [EarlyStoppingCallback(early_stopping_patience=3)]

You can change the patience here, hope it helps

Thank you very much Muhtasham Oblokulov!

1 Like