I’m facing a peculiar issue. I’m running evaluation on HANS. I have used this script more than a hundred times at this point but this error never appeared before.
When I use
run_hans.py, I’m getting this error in
RuntimeError: number of dims don't match in permute. I’ve inspected the dimension of input, it is
torch.Size). It is same as MNLI data sample from
run_glue which works. I tried with older versions of transformers, and get the same error.
I tried with other models such as
roberta-base and it works. I am not sure why this problem occurs because the attention weight vector has 4 dimensions
[# bs, n_heads, seq_length, 64]. Can anyone suggest a workaround to fix this ?