Does datasets support loading a file from a remote DVC repository, for instance versioned on Gitlab?
From DVC documentation in progress it seems that it does:
Hugging Face Datasets is a Python library accessing and sharing datasets. The DVC Python API provides DVCFileSystem, an fsspec-compatible filesystem which can be used with Hugging Face Datasets to load data from a DVC project. Please refer to the Hugging Face documentation for more detailed information on using fsspec filesystems with Hugging Face Datasets.
1 Like