Hi
during inference use output = model.generate(**batch) instead of output = model(**batch)
Also during training:
decoder_input_ids != target_data[“inputs_idx”]
labels = target_data[“inputs_idx”]
and decoder_input_ids = shift_to_right(target_data[“inputs_idx”]) - this action is performed automatically in library code, so you can simply omit decoder_input_ids argument