Hi, I have split 2 datasets of the same type, tweets and labels for sequence classification. Both I create the exact same way, from pandas datasets. They have the same columns, texts, labels in pre dataset conversion and later labels, input_ids and attention_masks. For one I can call Trainer.tr…

ValueError: Unable to create tensor for 1 dataset but not the other of same type

ollibolli March 23, 2022, 1:17pm 2

OMG I am the worst.
The labels in the other dataset were numbers as strings instead of ints.
Changing to int has worked.

Topic		Replies	Views
ValueError: Unable to create tensor, you should probably activate truncation and/or padding with ‘padding=True’ ‘truncation=True’ 🤗Transformers	1	836	November 22, 2023
ValueError: Unable to create tensor, you should probably activate truncation and/or padding with 'padding=True' 'truncation=True' to have batched tensors with the same length 🤗Transformers	4	37170	January 13, 2025
ValueError in using DataCollator: Unable to create tensor, you should probably activate truncation and/or padding with 'padding=True' 'truncation=True' to have batched tensors with the same length 🤗Transformers	1	7638	January 26, 2023
Trainer.train() padding error but it was working before 🤗Transformers	0	479	May 7, 2021
Issues with Data Collator and Tokenizing with NER Datasets 🤗Tokenizers	1	2530	May 9, 2022