BatchSampler - with trainer

Paul1020 · July 20, 2023, 11:16am

I am trying to use Transformers Trainer and I want to generate batches and not single items.

I need that in order to generate batches of different sizes without doing padding.
Is that possible to do with the implemented trainer ?

Topic		Replies	Views
Batch size for trainer.predict() 🤗Transformers	4	5658	November 26, 2022
How to print a few examples at the beginning of training when using Trainer? 🤗Transformers	5	1579	June 29, 2021
Expected input batch_size.. to match target batch_size 🤗Transformers	0	454	July 1, 2022
Reshaping logits when using Trainer Beginners	1	4188	May 23, 2022
Using Transformers with DistributedDataParallel — any examples? Intermediate	11	16670	May 8, 2023