How to distinguish train or eval in the callable function in dataset.map()?

🤗Datasets
1

In processing datasets, there is a little difference in training dataset and eval dataset. I want to use a parameter like mode=train/eval to implement the different processing method in a function, otherwise I have to write two functions which are mostly equal but different in some details, and pass them in train_dataset.map and eval_dataset.map seperately.

2

Hi! Yes, passing two different functions separately is one option. Another is to have one function (but still two map calls, one for each split) and pass different fn_kwargs in map to change the mode.

3

Thanks a lot! I missed the fn_kwargs and that is just what I want.