I am trying to find a workaround for my issue and to do so I would like to continue using the mapping function as I have a very large dataset (much bigger than the one in the linked post). I would like to store the multisets in a list of dicts like so: list_data = [] def _multisetify(code_file): …

How to edit outside variables in mapping function

mikehemberger August 13, 2024, 6:53pm 2

Hi again,
I suspect that this here and your other issue have the same solution.
Issue

Topic		Replies	Views
Error in Dataset Map Function Beginners	3	2165	March 22, 2023
Wondering if there is a way to modify the dataset directly? 🤗Datasets	0	20	January 3, 2025
Dataset map() raises value error when mapping list to dict-like class 🤗Datasets	6	87	August 15, 2024
Caching a dataset with map() when loaded with from_dict() 🤗Datasets	3	2660	March 22, 2023
How to apply .map() function and keep it as an iterator for a Hugging Face Dataset, in Streaming Mode without loading it to memroy? Beginners	1	384	May 21, 2024