While tokenizing the dataset I'm getting this type of error

[mwz]

can anyone help how to solve it…

[mineshj1291]

Hi,
I can’t see the error message. Can you paste the trace along with error message.

[mwz]

sorry there is no error message but process stops at 95%

I’m using this example notebooks/language_modeling_from_scratch.ipynb at main · huggingface/notebooks · GitHub

and change the dataset to OSCAR

[mariosasko]

Hi! This is a (known) bug in the printing logic, not in the processing itself (see Progress bars have color red and never completed to 100% · Issue #5117 · huggingface/datasets · GitHub), which we’ve already addressed. You can either install from “main” (pip install git+https://github.com/huggingface/datasets.git) or wait for the next release to fetch the fix.