Streaming support with download_custom

🤗Datasets
1

Do we not have support for streaming with dl_manger.download_custom() ?

>>> d = load_dataset("wdc.py", streaming=True)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ubuntu/environment/wdc/lib/python3.7/site-packages/datasets/load.py", line 1802, in load_dataset
    return builder_instance.as_streaming_dataset(split=split)
  File "/home/ubuntu/environment/wdc/lib/python3.7/site-packages/datasets/builder.py", line 1281, in as_streaming_dataset
    splits_generators = {sg.name: sg for sg in self._split_generators(dl_manager)}
  File "/home/ubuntu/.cache/huggingface/modules/datasets_modules/datasets/wdc/216e22ebfcc56acf151282dbaea6829607b49a41f953fa169b8f01fcc6b7f522/wdc.py", line 147, in _split_generators
    data_dir = dl_manager.download_custom(urls, self.config.s3fs.download)
AttributeError: 'StreamingDownloadManager' object has no attribute 'download_custom'
>>> AttributeError: 'StreamingDownloadManager' object has no attribute 'download_custom'
2

Hi ! No unfortunately, download_custom was only added for convenience.

I’d recommend you to move your data on HF if it’s possible, this way you can benefit from streaming for free