Hugging Face Forums

How to sample from the validation set when using Trainer?

🤗Transformers

brando October 5, 2023, 6:05pm 4

Questions:

will this ever be supported?
Why does the eval set have to be fixed?
How do we avoid memory issues because when the train set samples data to log and the eval data set samples to log I’ve gotten OM issues. Can they run seperately?

Evaluate subset of data during training

Topic		Replies	Views	Activity
Evaluate subset of data during training 🤗Transformers	5	5705	July 6, 2024
Use Trainer API with two valiation sets 🤗Transformers	1	1871	February 28, 2022
Technical clarification on the validation data vs. the training data in the trainer API 🤗Transformers	1	764	January 6, 2022
Adapt Decision Transformer collator to handle evaluation 🤗Transformers	1	234	July 13, 2024
Trainer will evaluate using my entire validation set(60k), which gives me cuda memory usage issue:. Is there a param that allows evaluating only on some batches in validation set? Beginners	4	1332	April 29, 2022