LayoutLMv3 processor error

John6666 · September 27, 2024, 8:48am

Apparently there is a precedent. It seems that the dataset and the model are incompatible. You will probably need to normalize the dataset manually.

github.com/microsoft/unilm

LayoutLM v2 on FUNSD-like dataset - index out of range in self

opened 04:56PM - 24 Jun 22 UTC

closed 04:50PM - 25 Jun 22 UTC

naourass

I'm using `transformers` to finetune ` microsoft/layoutlmv2-base-uncased` on my …custom dataset that is similar to FUNSD. After a few iterations of training I get this error : ``` Traceback (most recent call last): File "layoutlmV2/train.py", line 137, in <module> trainer.train() File "..../lib/python3.8/site-packages/transformers/trainer.py", line 1409, in train return inner_training_loop( File "..../lib/python3.8/site-packages/transformers/trainer.py", line 1651, in _inner_training_loop tr_loss_step = self.training_step(model, inputs) File "..../lib/python3.8/site-packages/transformers/trainer.py", line 2345, in training_step loss = self.compute_loss(model, inputs) File "..../lib/python3.8/site-packages/transformers/trainer.py", line 2377, in compute_loss outputs = model(**inputs) File "..../lib/python3.8/site-packages/torch/nn/modules/module.py", line 1131, in _call_impl return forward_call(*input, **kwargs) File "..../lib/python3.8/site-packages/transformers/models/layoutlmv2/modeling_layoutlmv2.py", line 1228, in forward outputs = self.layoutlmv2( File "..../lib/python3.8/site-packages/torch/nn/modules/module.py", line 1131, in _call_impl return forward_call(*input, **kwargs) File "..../lib/python3.8/site-packages/transformers/models/layoutlmv2/modeling_layoutlmv2.py", line 902, in forward text_layout_emb = self._calc_text_embeddings( File "..../lib/python3.8/site-packages/transformers/models/layoutlmv2/modeling_layoutlmv2.py", line 753, in _calc_text_embeddings spatial_position_embeddings = self.embeddings._calc_spatial_position_embeddings(bbox) File "..../lib/python3.8/site-packages/transformers/models/layoutlmv2/modeling_layoutlmv2.py", line 93, in _calc_spatial_position_embeddings h_position_embeddings = self.h_position_embeddings(bbox[:, :, 3] - bbox[:, :, 1]) File "..../lib/python3.8/site-packages/torch/nn/modules/module.py", line 1131, in _call_impl return forward_call(*input, **kwargs) File "..../lib/python3.8/site-packages/torch/nn/modules/sparse.py", line 158, in forward return F.embedding( File "..../lib/python3.8/site-packages/torch/nn/functional.py", line 2203, in embedding return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse) IndexError: index out of range in self ``` After further inspection (vocab size, bboxes, dimensions, classes...) I noticed that there's negative numbers inside the input tensor causing the error. These negative numbers are returned by _calc_spatial_position_embeddings(self, bbox) in modeling_layoutlmv2.py line 92 : `h_position_embeddings = self.h_position_embeddings(bbox[:, :, 3] - bbox[:, :, 1]) ` What does this line exactly do ? Assuming that the negative values are the cause of the issue, what could I do to prevent the input values from becoming negative ? Thanks in advance !

github.com/microsoft/unilm

LayoutLMv3: IndexError: index out of range in self on some inputs

opened 03:22PM - 09 Aug 22 UTC

closed 03:34PM - 03 Nov 22 UTC

HonzaCech

**Describe the bug** Model I am using (UniLM, MiniLM, LayoutLM ...): LayoutLM…v3 The problem arises when using: * my own modified scripts: (give details below) I'm using LayoutLMv3 from huggingface transformers - https://huggingface.co/docs/transformers/model_doc/layoutlmv3#transformers.LayoutLMv3Model - and I am getting an "IndexError: index out of range in self" on some inputs. It works fine for some inputs, but fails for others. I found a similar issue here -https://github.com/microsoft/unilm/issues/771 - but there, the problem was with bboxes. However, I am using the full processor with OCR that creates the bounding boxes for me, so my only actual input are PIL images. I initialize the processor with `processor = AutoProcessor.from_pretrained("microsoft/layoutlmv3-base", apply_ocr=True)` and the call it with `processor(img1, return_tensors="np", padding='max_length')` (img1 is PIL image) The full stack trace is ``` Traceback (most recent call last): File "/home/h/projects/layoutLM/main_layoutLM.py", line 58, in <module> main() File "/home/h/projects/layoutLM/main_layoutLM.py", line 37, in main output1, output2 = net(img0, img1) File "/home/h/projects/layoutLM/venv/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl return forward_call(*input, **kwargs) File "/home/h/projects/layoutLM/layoutlm_classifier_contrastive.py", line 36, in forward output1 = self.forward_once(input1) File "/home/h/projects/layoutLM/layoutlm_classifier_contrastive.py", line 29, in forward_once output = self.layout_LM(**x) File "/home/h/projects/layoutLM/venv/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl return forward_call(*input, **kwargs) File "/home/h/projects/layoutLM/venv/lib/python3.9/site-packages/transformers/models/layoutlmv3/modeling_layoutlmv3.py", line 833, in forward embedding_output = self.embeddings( File "/home/h/projects/layoutLM/venv/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl return forward_call(*input, **kwargs) File "/home/h/projects/layoutLM/venv/lib/python3.9/site-packages/transformers/models/layoutlmv3/modeling_layoutlmv3.py", line 261, in forward position_embeddings = self.position_embeddings(position_ids) File "/home/h/projects/layoutLM/venv/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl return forward_call(*input, **kwargs) File "/home/h/projects/layoutLM/venv/lib/python3.9/site-packages/torch/nn/modules/sparse.py", line 158, in forward return F.embedding( File "/home/h/projects/layoutLM/venv/lib/python3.9/site-packages/torch/nn/functional.py", line 2199, in embedding return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse) IndexError: index out of range in self Process finished with exit code 1 ``` where `output = self.layout_LM(**x)` is the actual calling of LayoutLM, which is created by `self.layout_LM = AutoModel.from_pretrained("microsoft/layoutlmv3-base")` I have no idea what could be wrong, as I said, it works fine for some inputs and I don't see anything special about cases where it fails - Platform: Ubuntu 20.04 - Python version: 3.9 - PyTorch version (GPU?): 1.12.0+cpu

Topic		Replies	Views
[LayoutLMv3] index out of range in self inside outputs = model(**encoding) Models	4	2737	May 10, 2024
Index out of range layoutlm Beginners	5	1940	March 10, 2021
LayoutXLM training - index out of bounds: 0 <= tmp30 < 1L Beginners	0	10	September 3, 2024
Error while using LILT model "index out of range in self" 🤗Transformers	5	703	March 14, 2024
LayoutLMv3 inference - bboxes are incorrect 🤗Transformers	0	116	May 10, 2024

LayoutLMv3 processor error

Related topics