I believe this is the same problem as in this topic. Make sure your model outputs tuples if you want to use it with Trainer. And your compute_loss should have a return_outputs argument to work well with the last version of transformers.
1 Like