Hi,
So I was trying out invoice information extraction using LayoutLMV3. There are total 11 ner present. However all of them are not present in every dataset. Like for example:
image_1 : has 10 labels(out of the above told 11)
image_2 : has 11 labels and so on.
The problem is that due to this inconsistency in labels when I give batch_size more than 1 than the following error pops up:
ValueError: expected sequence of length 509 at dim 1 (got 512)
For few dataset I observed that the labels shape is:
torch.Size([509]) and for few it’s labels torch.Size([512]).
Any help on how to solve this is highly appreciated.
Regards
The problem was with the labeling. While labeling the ner corresponding to bbox and tokens few were left out. Thus, the mismatch.