I am training a Swinv2 model on a multi-class problem, default loss (I’m using Trainer, no custom). And I got a suspicious case of having a constant loss for 3 epochs, then a sharp decrease, and a new stagnation before the early stop kicked it:
At first, I’d say incorrect loss, but it’s the default one. Or incorrect reporting, but it’s effectively different epochs. The model seems ok from a results point of view.
Would you have any idea what would cause anything like that?