Dataset Viewer for dataset with downloadable data

Hi :hugs:

I am not hosting my files on huggingface, but I am downloading a .zip file from my university dataset hub. Further, the files are processed in the _generate_examples functions of the GeneratorBasedBuilder. How can I make this compatible with the dataset viewer? It’s about: UKPLab/PeerQA · Datasets at Hugging Face .

1 Like

Hi ! unfortunately the viewer doesn’t support datasets based on python scripts (for obvious security reasons)

Feel free to host the dataset as Parquet files using ds.push_to_hub() instead, this way you’ll get a viewer

1 Like

Thanks for clarifying, @lhosteq! I see, it makes sense that the code cannot be executed. The code is required because it needs to download some and process some data that can’t be hosted on the hub due to licenses.
I have now added the permissible part using ds.push_to_hub(). I can see the parquet files have been uploaded, but the viewer is still showing that it cant be shown due to the remote code. Does it mean that the viewer only works if there is no loading script present?

1 Like

It seems having both parquet files and a loading script is not supported, meaning it will not enable the dataset viewer (Enabling dataset viewer by coexistence of loading script and parquet files - #5 by lhoestq). So i removed the parquet files again.

1 Like