I am modifying components of a GPT2 like model and am wondering what HF Dataset would be the best to debug my operations. Ultimately I am using OpenWebText and or Wiki103 but these take a long time to parse and batch correctly before the model actually trains on any data.
What are really small HF datasets that would work well for this?
I have taken a bit of a look but nothing jumps out.
Thank you.