Where to upload big datasets for free?

Hi all!
I’m going to release a huge dataset (about 120 GB) of text to the :hugs: / Datasets which I believe will help the Persian nlp community a lot. I haven’t tried uploading the dataset directly to the hub, it may be possible to do so by git-lfs but I’m not sure about it.

Is there any free hosting service that I can upload the big .txt file into it and then use dataset scripts to refer to it?

2 Likes

Hi ! You can host your text files for free on the Hugging Face Hub, just create a dataset repository and upload your files there. You can use the website to do so, or use git or python.

You can check the docs for more info : Share a dataset to the Hub

2 Likes

Thanks for that. I’ll check it and close this topic as solved when I upload my dataset.

Please post a link to the dataset once posted. I’m curious to see what it can do for the Persian NLP community.

2 Likes

As an update these links helps me a lot sovling my issues:

1 Like

As I promised to you, here are uploaded materials: