Creating a HF Dataset from lakeFS with S3 storage takes too much time!

@Adam-Ben-Khalifa you can try to load the data in streaming mode, also after you converted the data into the datasets library consider saving it locally or pushing it to the hub

2 Likes