Best dataset for debugging a language model?

I am modifying components of a GPT2 like model and am wondering what HF Dataset would be the best to debug my operations. Ultimately I am using OpenWebText and or Wiki103 but these take a long time to parse and batch correctly before the model actually trains on any data.

What are really small HF datasets that would work well for this?

I have taken a bit of a look but nothing jumps out.

Thank you.