Hi All, Just want to know, is there any way to batch decode variable length sentences. For example [S1, S2] , where S1 has 5 words abd S2 has 10 words . Can we decode it using GPT2 , BERT etc?

Variable length batch decoding

fzyzcjy March 31, 2024, 1:27pm 12

Hi, is there any updates? Thanks!

Topic		Replies	Views
Decoder generate with prompts of variable lengths? 🤗Transformers	0	664	May 25, 2022
Issue with Decoding in HuggingFace 🤗Tokenizers	2	3878	March 24, 2022
Batch generation with GPT2 🤗Transformers	12	17189	January 16, 2024
Parallelize Mistral/ llama2 output 🤗Transformers	1	154	May 25, 2024
Llama2 pad token for batched inference Models	7	15664	March 31, 2024