Generation but constraining first few tokens

🤗Transformers
1

I’m trying to use a pre-trained T5ForConditionalGeneration model to do generation from a prefix of a sentence. For example, I might feed it “ 2 + 2 =”, and I’d like it to generate “4”.

I notice that the generate() function for this model always produces a generation starting with the two tokens [pad_token, bos_token]. Therefore, if I just pass the input through model.generate, I get a nonsensical output. I see two potential solutions:

  1. Feed it in “”, and fix the first few tokens of the generation to be “2 + 2 =”
  2. Feed it in “ 2 + 2 =”, and find a way to remove the pad and bos tokens from the generation.

I don’t know how to do either, and help would be appreciated! Thanks!