Hello, does anyone know how to get a train Dataloader from a Trainer? I need it to compute some metrics during training process on a training dataset but I don’t want to initialize Dataloader twice due to memory limits.
Or does transformers Trainer us it’s own data storage structures? It would be great to hear some advices!
Well, I found it. I`m using on_evaluate() callback and train_dataloader parameter is contained in the **kwargs argument of this function.
Answering here as I ask myself the same question few days ago:
Trainer does indeed provide a way to retrieve the dataloader(s), actually it even provide three methods:
trainer.get_train_dataloader()trainer.get_eval_dataloader()trainer.get_test_dataloader()By utilizing these methods you can access the dataloaders without needing to initialize new ones.
I hope this response (even late) proves itself helpful to someone ^^