@panigrah thank you very much. Maybe you also know if it’s possible to download a dataset in multi-processed way? For some reason setting num_proc does not work at all… My dataset has 58 parquet files and i was hoping passing num_proc to load_dataset would spawn 58 Python processes each downloading its own parquet so I can load my dataset in 1 minutes instead of 50…