Hello,
I’d like to use my own ModelOutput class, which has multiple losses as shown below.
loss is the total loss, and there are two losses that are weighted and added.
I want to show the two losses besides the total loss, but when I use Trainer with this original class, it failed during the evaluation step.
Is there anything I need to be aware of when defining this class?
definition of my own ModelOutput class
@dataclass
class SequenceClassifierMultiLossOutput(ModelOutput):
"""
Base class for outputs of sentence classification models.
Args:
loss (:obj:`torch.FloatTensor` of shape :obj:`(1,)`, `optional`, returned when :obj:`labels` is provided):
Classification (or regression if config.num_labels==1) loss.
logits (:obj:`torch.FloatTensor` of shape :obj:`(batch_size, config.num_labels)`):
Classification (or regression if config.num_labels==1) scores (before SoftMax).
hidden_states (:obj:`tuple(torch.FloatTensor)`, `optional`, returned when ``output_hidden_states=True`` is passed or when ``config.output_hidden_states=True``):
Tuple of :obj:`torch.FloatTensor` (one for the output of the embeddings + one for the output of each layer)
of shape :obj:`(batch_size, sequence_length, hidden_size)`.
Hidden-states of the model at the output of each layer plus the initial embedding outputs.
attentions (:obj:`tuple(torch.FloatTensor)`, `optional`, returned when ``output_attentions=True`` is passed or when ``config.output_attentions=True``):
Tuple of :obj:`torch.FloatTensor` (one for each layer) of shape :obj:`(batch_size, num_heads,
sequence_length, sequence_length)`.
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.
loss_one (:obj:`torch.FloatTensor` of shape :obj:`(1,)`, `optional`, returned when :obj:`labels` is provided):
first (or regression if config.num_labels==1) loss.
loss_two (:obj:`torch.FloatTensor` of shape :obj:`(1,)`, `optional`, returned when the second loss is used):
second (or regression if config.num_labels==1) loss.
"""
loss: Optional[torch.FloatTensor] = None
logits: torch.FloatTensor = None
hidden_states: Optional[Tuple[torch.FloatTensor]] = None
attentions: Optional[Tuple[torch.FloatTensor]] = None
loss_one: Optional[torch.FloatTensor] = None
loss_two: Optional[torch.FloatTensor] = None
error message
File "/****/.pyenv/versions/anaconda3-2020.07/lib/python3.8/site-packages/transformers/trainer.py", line 844, in train
self._maybe_log_save_evaluate(tr_loss, model, trial, epoch)
File "/****/.pyenv/versions/anaconda3-2020.07/lib/python3.8/site-packages/transformers/trainer.py", line 906, in _maybe_log_save_evaluate
metrics = self.evaluate()
File "/****/.pyenv/versions/anaconda3-2020.07/lib/python3.8/site-packages/transformers/trainer.py", line 1323, in evaluate
output = self.prediction_loop(
File "/****/.pyenv/versions/anaconda3-2020.07/lib/python3.8/site-packages/transformers/trainer.py", line 1447, in prediction_loop
preds_host = logits if preds_host is None else nested_concat(preds_host, logits, padding_index=-100)
File "/****/.pyenv/versions/anaconda3-2020.07/lib/python3.8/site-packages/transformers/trainer_pt_utils.py", line 84, in nested_concat
return type(tensors)(nested_concat(t, n, padding_index=padding_index) for t, n in zip(tensors, new_tensors))
File "/****/.pyenv/versions/anaconda3-2020.07/lib/python3.8/site-packages/transformers/trainer_pt_utils.py", line 84, in <genexpr>
return type(tensors)(nested_concat(t, n, padding_index=padding_index) for t, n in zip(tensors, new_tensors))
File "/****/.pyenv/versions/anaconda3-2020.07/lib/python3.8/site-packages/transformers/trainer_pt_utils.py", line 86, in nested_concat
return torch_pad_and_concatenate(tensors, new_tensors, padding_index=padding_index)
File "/****/.pyenv/versions/anaconda3-2020.07/lib/python3.8/site-packages/transformers/trainer_pt_utils.py", line 47, in torch_pad_and_concatenate
if len(tensor1.shape) == 1 or tensor1.shape[1] == tensor2.shape[1]:
IndexError: tuple index out of range
Thank you in advance.