Is Eval and Validation same in Trainer API?

Hi, I am a bit confused if the eval dataset parameter is used during the training.

#Trainer itself.
trainer = Trainer(
    data_collator = data_collator_

is the eval_dataset only used when we do trainer.evaluate() ?

Yes, it’s the default dataset used for that method (which will be used if you pass an eval_strategy to evaluate every epoch or n steps).

1 Like

Okay perfect, then i will not put my test set over there Thanks! @sgugger

Although i want to get the f1 score of my model on the test set, do you know if there is a metric api from hugginface i could use?

If so could you please link me a small script?

You can look at any of the examples or the course section on the Trainer on how to du this using a compute_metrics function.

1 Like

Huge thanks again!

Have a nice day

1 Like