Key Error ‘loss’ - finetuning [ arabert , mbert ]

Beginners
1

Hello all!
I am trying to fine-tune mbert and arabert models on translation task as explained here, however, I am getting this error Key Error ‘loss’
the input to the model :

DatasetDict({
    train: Dataset({
        features: ['attention_mask', 'input_ids', 'labels', 'src', 'token_type_ids', 'trg'],
        num_rows: 40
    })
    validation: Dataset({
        features: ['attention_mask', 'input_ids', 'labels', 'src', 'token_type_ids', 'trg'],
        num_rows: 11
    })
})

Note: when I run trainer.train() , it returns a message like The following columns in the training set don't have a corresponding argument in BertModel.forward and have been ignored: trg, labels, src.

any hints?

2

You can use the Trainer on BertModel as it has no objective, it’s the body of the model with no particular head. You should pick a model with a head suitable for the task at hand.