I have a text classification problem. I am using transformers for predictions. Output probabilities are really skewed:
Is there a way to make them more equally distributed?
Is the figure you referenced your output probability? I’m confused. You haven’t mentioned the distribution of your training or test data.
If the test data is skewed, and models are predicting correctly the output probability distribution will also be skewed. If the image you gave is what you call output probability, it may be that you’re using some thresholding.
@Sandy1857 Thanks for the reply.
Both training and test data are perfectly balanced 50% class 0 and 50% class 1.
The model produces probabilities like:
3.6716461181640625e-05,
3.647804260253906e-05,
3.6835670471191406e-05,
3.6597251892089844e-05,
3.7670135498046875e-05,
0.9999595880508423,
0.9999598264694214,
0.9999594688415527,
0.9999581575393677,
0.9999593496322632
The right way is to change the training data. With this dataset, the model is really sure it can discriminate between the two classes.
I am wondering if ther is a way to smooth the histogram: show more probabilities like 0.1 0.2 0.3 .. I would like to use a parameter at training time to do this.
But isn’t a good discrimination ability of a model is what we need. Your model seems to be doing a great job at that. What purpose would be served by decreasing this ability?
Anyway you could look at label smoothing in cross entropy loss.
Thanks.
Yes, true. Good discrimination ability is what we need. But I don’t think that I have the right data now. This data is too easy to discriminate. I am testing it on a few external samples and it does not work. E.g. I get probability 0.9999 for a sample of class 0. If I had more equally distributed probabilities, at least I could play with thresholds.
Label smoothing sounds like a good parameter to play with!