Does batch mapping work properly for IterableDatasets? I have my processing code set up to return for each column a list of rows, but it seems to only ignore all other entries in the list except the first entry. labels_ids = [reasoning_labels, answer_labels] return { 'la…

Fixed. Turns out I had to remove all my original columns

One-to-many batch mapping with IterableDatasets and batch_size=1 doesn't work

system Closed April 14, 2025, 7:49pm 3

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.

Topic		Replies	Views
How to use `map` or similar when one row is mapped to multiple rows? 🤗Datasets	1	2794	July 20, 2021
Clarification on Batch mapping 🤗Datasets	2	899	November 2, 2023
Iterable of batches from IterableDataset 🤗Datasets	1	162	September 24, 2024
Tokenize iterable dataset Models	0	262	June 7, 2023
Dataset.map() with batching and multiprocessing 🤗Datasets	1	280	March 5, 2024