Is it possible to use multithreading with dataset.map(...)
? I’m currently using multiprocessing via num_proc
, but it’s too slow as my processing function makes Internet requests.
Using more threads via multithreading could speed it up, no?
cc @lhoestq
1 Like
Hi ! I think you’d need to implement the multithreading in the function you pass to map