Load_dataset is very slow

Beginners
1

i have a big problem load_dataset indexing is very slow ex) x[“text”][idx] or x[“audio”][idx][‘array’] this line take 30 second what should i do? already keep_in_memory is True

2

@cogns3942 Try x[idx]["text"] instead, it should be faster.

My guess is that when you request x["text"][idx], the entire list x["text"] has to be brought into memory which makes it slower.

1 Like
3

oh my god thank you I was using it wrong

4

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.