Using IterableDataset with Trainer - `IterableDataset' has no len()

[marshmellow77]

Hi Eric - you need to format your dataset for PyTorch first, like so: torch_iterable_dataset = dataset.with_format("torch").

More info on how and why can be found here: Error iteration over IterableDataset using Torch DataLoader · Issue #2583 · huggingface/datasets · GitHub

Hope that helps!

Cheers
Heiko