My classification dataset has a extrem minority classes with only two observations total:
Counter({
7: 3561,
6: 811,
8: 408,
2: 175,
9: 122,
10: 106,
4: 98,
0: 36,
3: 34,
14: 30,
12: 28,
5: 27,
13: 13,
1: 13,
11: 2})
When applying a train-test split via
ds = ds.train_test_split(test_size=0.1, stratify_by_column="label")
the function prioritizes the train/test-ration over the stratification.
I.e. both data points for class 11 are in the training set and none in the test set.
This is obviously to retain the split ration but I specifically want to test the performance of my model in this low availability cases.
In this example I could manually move the sample via ds.select() and ds.add_item() for this one class.
But would there be better way to “force” stratification when for example faced with multiple of such minority classes in a dataset?