Dataset not downloading

Hi there,
I am trying to create a new dataset: mehdie/sefaria 路 Datasets at Hugging Face
When I try to use it to train a tokenizer, the data itself (data directory) does not get downloaded. I get 0 files and 0 records. Did I do something wrong? I have a custom _split_generators function. Maybe something needs to be done there?

Hi ! In your script you seem to use glob but it鈥檚 not necessary.

You can use to download a list of parquet files, and it will return the list of downloaded parquet file paths

The function returns a list of text files that are around 132bytes and contain Things like this:

oid sha256:08f7e86610f17d1addd6999e39c969aea926902d0cf616e8d5bd90f41ba124d1
size 7518478

Interestingly, in the same directory each of these files is accompanied by a file with the same name and a .json extension that contains the URL to one of my missing files:

{"url": "", "etag": null}

The URLs were not pointing to the file itself, but to its git lfs metadata file. You should use

instead of