In most of the cases I see the use of a mean pooling being taken, even in the case of a loss function, the values are averaged, but if the values that are being averaged, have outliers, like,
[1, 1.1, 0.9, 1.2, 10000]
then the average would be influenced by the high term (
this problem would also be there if we take the sum.
one way is then to take the median, but if we take the median, then there is a strong chance of loss/pooled output coming out to be zero if more than half the values are zero, that is in a case like,
[0, 0, 0, 0, 0, 0, 2, 3, 4, 5, 8]
the median would be zero.
there appears to be issues with all three sum, average, median.
Is there an alternative to these three?