Hugging Face Forums

How to sample from the validation set when using Trainer?

🤗Transformers

brando October 5, 2023, 6:05pm 4

Questions:

will this ever be supported?
Why does the eval set have to be fixed?
How do we avoid memory issues because when the train set samples data to log and the eval data set samples to log I’ve gotten OM issues. Can they run seperately?

Evaluate subset of data during training

Topic		Replies	Views	Activity
Evaluate subset of data during training 🤗Transformers	6	5915	November 21, 2025
Use Trainer API with two valiation sets 🤗Transformers	1	1919	February 28, 2022
Technical clarification on the validation data vs. the training data in the trainer API 🤗Transformers	1	782	January 6, 2022
Using Seq2SeqTrainer to eval during training? 🤗Transformers	1	1063	November 27, 2021
Trainer will evaluate using my entire validation set(60k), which gives me cuda memory usage issue:. Is there a param that allows evaluating only on some batches in validation set? Beginners	4	1348	April 29, 2022