Could I inference the Encoder-Decoder model without specify "decoder_input_ids"?

yurii · April 30, 2021, 7:03pm

In case of training conditional model (e.g BartForConditionalGeneration), when decoder_input_ids is absent it will be created automatically by right-shift of labels:

github.com

huggingface/transformers/blob/8d43c71a1ca3ad322cc45008eb66a5611f1e017e/src/transformers/models/bart/modeling_bart.py#L1283-L1287


if labels is not None:
    if decoder_input_ids is None:
        decoder_input_ids = shift_tokens_right(
            labels, self.config.pad_token_id, self.config.decoder_start_token_id
        )

In case of training bare model (e.g BartModel), when decoder_input_ids is absent it will be created automatically by right-shift of input_ids:

github.com

huggingface/transformers/blob/8d43c71a1ca3ad322cc45008eb66a5611f1e017e/src/transformers/models/bart/modeling_bart.py#L1147-L1152


# different to other models, Bart automatically creates decoder_input_ids from
# input_ids if no decoder_input_ids are provided
if decoder_input_ids is None and decoder_inputs_embeds is None:
    decoder_input_ids = shift_tokens_right(
        input_ids, self.config.pad_token_id, self.config.decoder_start_token_id
    )

By the way, try
"inputs_ids": source_data["inputs_ids"]
instead of
"inputs_idx": source_data["inputs_idx"]

Topic		Replies	Views
The meaning of 'decoder input ids' in encoder-decoder model Beginners	1	2408	July 29, 2022
Is there a way to return the "decoder_input_ids" from "tokenizer.prepare_seq2seq_batch"? 🤗Transformers	5	3353	December 29, 2020
From Transformers Version v4.12.0 onwards, The example colab BERT2BERT is wrong. (Things to keep in mind when using from transformers import EncoderDecoderModel) 🤗Transformers	0	272	February 16, 2024
Seq2Seq Loss computation in Trainer Beginners	9	6039	October 28, 2021
T5 fine tuning, loss difference when using labels and decoder_input_ids 🤗Transformers	2	1178	October 12, 2020

Could I inference the Encoder-Decoder model without specify "decoder_input_ids"?

Related topics