Behavior of shuffled parquet dataset

🤗Datasets
1

if a dataset is stored as parquets, loading with huggingface load_dataset, and then shuffled, does this mean that batches contain rows from several files? Or does it only shuffle the order of reading parquet files?

1 Like