Do I need to apply the softmax function to my logit before calculating the CrossEntropyLoss?

🤗Transformers
1

Hello, I am trying to compute the CrossEntropyLoss directly by using this code:

loss_fct = CrossEntropyLoss()
mc_loss = loss_fct(reshaped_logits, mc_labels)

If the reshaped_logits contain the logit values before softmax, should I apply nn.softmax function before I do loss_fct(reshaped_logits, mc_labels)? Thank you,

2

PyTorch CrossEntropyLoss combines softmax and the cross-entropy loss, so no.

1 Like