For the logits from HuggingFace Transformer models, can the sum of the elements of the logit vector be greater than 1?

🤗Transformers
1

Hello,
For the logits from HuggingFace Transformer models, can the sum of the elements of the logit vector be greater than 1?
I am getting a logit vector which their element sums to a number greater than 1, and I just think this is strange.

Thanks,

2

This is normal. The logits are not normalized, and must be passed through a softmax in order to form a valid probability distribution over tokens.

2 Likes