Hello everyone
I trained a pretrained model and add others layer and train them.
During the training the loss goes from 5 to [0.6,0.8]
So it seems that the model converge.
But then after the training I test the model and it generate no sens output.
The loss is calculated as usual but as well as in mixtral I add an aux_loss (calculated with same function as mixtral)
I don’t understand why at the same time the loss is low and the output is incorrect.
Best regards