In most of the cases I see the use of a mean pooling being taken, even in the case of a loss function, the values are averaged, but if the values that are being averaged, have outliers, like,
[1, 1.1, 0.9, 1.2, 10000]
then the average would be influenced by the high term (10000 here)
this problem would also be there if we take the sum.
one way is then to take the median, but if we take the median, then there is a strong chance of loss/pooled output coming out to be zero if more than half the values are zero, that is in a case like,
[0, 0, 0, 0, 0, 0, 2, 3, 4, 5, 8]
the median would be zero.
there appears to be issues with all three sum, average, median.
Is there an alternative to these three?