Hi All, Just want to know, is there any way to batch decode variable length sentences. For example [S1, S2] , where S1 has 5 words abd S2 has 10 words . Can we decode it using GPT2 , BERT etc?

Variable length batch decoding

valhalla October 4, 2020, 4:16pm 2

What do you mean by decoding, as in decoding the generated tokens by GPT2 or making predictions on a batch of sequences ?

Topic		Replies	Views
Decoder generate with prompts of variable lengths? 🤗Transformers	0	664	May 25, 2022
Issue with Decoding in HuggingFace 🤗Tokenizers	2	3878	March 24, 2022
Batch generation with GPT2 🤗Transformers	12	17190	January 16, 2024
Parallelize Mistral/ llama2 output 🤗Transformers	1	154	May 25, 2024
Llama2 pad token for batched inference Models	7	15664	March 31, 2024