Question about all_head_size under BertSelfAttention

🤗Transformers
1

While going through the codebase , I found the following code under the BertSelfAttention class here - (BertSelfAttention-

self.num_attention_heads = config.num_attention_heads
self.attention_head_size = int(config.hidden_size / config.num_attention_heads)
self.all_head_size = self.num_attention_heads * self.attention_head_size

I am not sure I understand the reason for calculating all_head_size again instead of assigning it the value of config.hidden_size directly.

Am I missing something?

I also see a check right above this that ensures that the hidden_size is divisible by num_attention_heads:

    if config.hidden_size % config.num_attention_heads != 0 and not hasattr(config, "embedding_size"):
        raise ValueError(
            "The hidden size (%d) is not a multiple of the number of attention "
            "heads (%d)" % (config.hidden_size, config.num_attention_heads)
        )