Variable length batch decoding

[valhalla]

There is no pad token for GPT, you can manually set the eos token as pad token using

tokenizer.pad_token = tokenizer.eos_token