I am using load_dataset to load the new Wikipedia dataset by:
load_dataset(“wikipedia”, language=“en”, date=“20230301”, beam_runner=“DirectRunner”)
I have successfully download the whole data, however, the procedure gets stuck after downloading it and I have to interrupt it:
Any solutions?