I am trying to build a dataset using load_dataset for a classification task. The labels are encoded on the parent folder (like /label/file_1.txt). How can I create that new column with the label? Thanks.
PS: my whole folder structure is (train/valid) / (label) / file_name.txt so It would be nice if I could get the splits from the grandfather folder.