How to get a Dataloader from a Trainer?

Hello, does anyone know how to get a train Dataloader from a Trainer? I need it to compute some metrics during training process on a training dataset but I don’t want to initialize Dataloader twice due to memory limits.

Or does transformers Trainer us it’s own data storage structures? It would be great to hear some advices!

Well, I found it. I`m using on_evaluate() callback and train_dataloader parameter is contained in the **kwargs argument of this function.

Answering here as I ask myself the same question few days ago:

Trainer does indeed provide a way to retrieve the dataloader(s), actually it even provide three methods:

  • trainer.get_train_dataloader()
  • trainer.get_eval_dataloader()
  • trainer.get_test_dataloader()

By utilizing these methods you can access the dataloaders without needing to initialize new ones.
I hope this response (even late) proves itself helpful to someone ^^

1 Like