I’ve modeled my training script on the information in the finetuning with custom datasets documentation (https://huggingface.co/transformers/custom_datasets.html).
I have both a custom dataset and a custom model (I used the run_language_modeling.py script to pretrain the roberta-base model with our raw texts).
when I run trainer.train() I get the error: ValueError: Expected input batch_size (16) to match target batch_size (64), when the model is computing the loss on a training_step
I don’t know where target batch_size is being set. The input batch_size matches the value I have for per_device_train_batch_size.
Sorry I know this is an old post, but did you manage to resolve this? I’ve got the same issue when using DistilBert using custom dataset as in their tutorial.
ValueError: Expected input batch_size (16) to match target batch_size (2848).
Sorry. I did resolve it, but have no memory of how. I’m up to using transformers 4.9.2 now and do not have the issue and do not need to make changes to their run_classification script. (I’m using the pytorch version).
In my case it was because I was trying to train as a multi-label classification model by encoding the labels with sklearn’s MultiLabelBinarizer but forgot to set the config parameter for multilabel setting to the model:
