Resuming run: resume dataloader at specific index

When resuming a run, the dataloader restarts from the beginning. Is there a simple way to make it resume from the index at which we finished the previous run?

No, that’s not baked in in Accelerate, could be a nice feature request (but doing it efficiently will require a bit of time :sweat_smile: ). For now you should manually skip the first batches if it’s really important to you (if you’re using iterable datasets, we won’t be able to do anything more efficient anyway).

1 Like