Hello, everyone
Since the preprocessing of my task dataset takes a lot of CPU time, I wanted to not spend a lot of time processing data before each epoch, so I found the MultiEpochsDataLoader in timm, which significantly reduced my epoch initialization time.
When I used the MultiEpochsDataLoader with the Accelerator, it was still able to speed up the epoch initialization time significantly, but I found that the memory grew significantly after each epoch until it blew up.
My accelerate=1.1.1 and timm=1.0.7.
Hope to get help, thank you!