Early stopping + trainer + hub

🤗Transformers
1

Hi! I am trying to fine-tune a model with early stopping using trainer and then publish it on the hub. However, from the automatically created model card, it looks like the updated model is the last one and not the best one.

See, for example, this model.

The arguments I use are:

training_args = TrainingArguments(
        output_dir="irony_"+language+"_"+"_".join(nationality.split()),          
        num_train_epochs=20,  
        learning_rate = 5e-06,
        per_device_train_batch_size=16,  
        per_device_eval_batch_size=64,   
        evaluation_strategy="epoch",
        save_strategy="epoch",
        save_total_limit=2,
        logging_strategy="epoch",
        overwrite_output_dir = True,
        load_best_model_at_end = True,
    )

I then train using the trainer, and push to the hub using

    trainer.push_to_hub("irony_"+language+"_"+"_".join(nationality.split()))

I use transformers 4.34.1

2

Thanks for your question @silvia-casola!

Pinging @muellerzr and @smangrul for advice

1 Like
3

Hi @muellerzr @smangrul , any advice?

4

The trainer pushes the best model, but the automatically-created card reports the performance of the last one.

Check this: Clarification on push_to_hub, best model, and model card