How to sample from the validation set when using Trainer?

brando · October 5, 2023, 6:05pm

Questions:

will this ever be supported?
Why does the eval set have to be fixed?
How do we avoid memory issues because when the train set samples data to log and the eval data set samples to log I’ve gotten OM issues. Can they run seperately?

Topic		Replies	Views
Evaluate subset of data during training 🤗Transformers	4	3356	October 5, 2023
Is Eval and Validation same in Trainer API? Beginners	4	1275	September 14, 2021
Trainer will evaluate using my entire validation set(60k), which gives me cuda memory usage issue:. Is there a param that allows evaluating only on some batches in validation set? Beginners	4	955	April 29, 2022
2 evalset but got no validation loss Beginners	4	116	April 17, 2024
Technical clarification on the validation data vs. the training data in the trainer API 🤗Transformers	1	517	January 6, 2022