How to ensure the dataset is shuffled for each epoch using Trainer and Datasets?

[sgugger]

@dashapyly

Is there a parameters that controls whether or not the data get reshuffled before each epoch?

No, this is would be very bad practice so we don’t offer that option.

@dashapyly

And whether or not it is grouped by length?

That would be the group_by_length argument.

@dashapyly

Additionally, if the training is aborted and I’m restarting from a checkpoint - does the checkpoint have information about the shuffling order for this given epoch and which datapoints still haven’t gone through this epoch already?

Yes training will resume with the same shuffle, at the same point you were at the time of the save.