Unable to create tensor, you should probably activate truncation and/or padding with 'padding=True' 'truncation=True' to have batched tensors with the same length

Kowsher · October 22, 2022, 7:06pm

To train a Bert LM, I updated the DataCollatorForLanguageModeling function, and it creates another input_ids as new_input_ids which is the same size as input_ids. To train bert model it accepts input_ids and new_input_ids in BertForPreTraining class. But facing issues in batch size, getting error
as ’ Unable to create tensor, you should probably activate truncation and/or padding with ‘padding=True’ ‘truncation=True’ to have batched tensors with the same length’
I get this issue, comes from this 699 line of this file

github.com

huggingface/transformers/blob/bd9871657bb9500a9f4437a873db6df5f1ae6dbb/src/transformers/tokenization_utils_base.py#L715


      
                          # elif tensor.ndim < 2:
                          #     tensor = tensor[None, :]
          
          
                self[key] = tensor
                  except:  # noqa E722
                      if key == "overflowing_tokens":
                          raise ValueError(
                              "Unable to create tensor returning overflowing tokens of different lengths. "
                              "Please see if a fast version of this tokenizer is available to have this feature available."
                          )
                      raise ValueError(
                          "Unable to create tensor, you should probably activate truncation and/or padding "
                          "with 'padding=True' 'truncation=True' to have batched tensors with the same length."
                      )
          
          
    return self
          
          
@torch_required
          def to(self, device: Union[str, "torch.device"]) -> "BatchEncoding":
              """
              Send all values to device by calling :obj:`v.to(device)` (PyTorch only).

Now how can i keep the same size of new_input_ids same as input_ids for every batch

Topic		Replies	Views
Key error: 0 in DataCollatorForSeq2Seq for BERT Beginners	10	3444	March 13, 2024
Bert strugling with Padded sentence 🤗Transformers	0	345	August 24, 2021
Why is the tensor produced by inference so big? Beginners	2	347	April 17, 2023
Missing, yet not missing, input_ids 🤗Transformers	0	683	March 16, 2023
How to make Data Loader for "Multi-Head" Regression which can be used with Trainer 🤗Transformers	0	175	December 24, 2023

Unable to create tensor, you should probably activate truncation and/or padding with 'padding=True' 'truncation=True' to have batched tensors with the same length

Related Topics