Is it possible to use multithreading with dataset.map(...)? I’m currently using multiprocessing via num_proc, but it’s too slow as my processing function makes Internet requests.
Using more threads via multithreading could speed it up, no?
cc @lhoestq ![]()
1 Like
Hi ! I think you’d need to implement the multithreading in the function you pass to map