Multithreading with map

Is it possible to use multithreading with dataset.map(...)? I’m currently using multiprocessing via num_proc, but it’s too slow as my processing function makes Internet requests.
Using more threads via multithreading could speed it up, no?
cc @lhoestq :hugs:

1 Like

Hi ! I think you’d need to implement the multithreading in the function you pass to map