Mapping large datasets

@lhoestq Yes, I agree. We can set the default value of writer_batch_size to None in the signature of map, and do:

writer_batch_size = batch_size if writer_batch_size is None else writer_batch_size

in the body.

1 Like