Hi All, Just want to know, is there any way to batch decode variable length sentences. For example [S1, S2] , where S1 has 5 words abd S2 has 10 words . Can we decode it using GPT2 , BERT etc?

Variable length batch decoding

s4sarath October 5, 2020, 1:18pm 11

I have implemented in TF2.0 . I had to make quite a bunch of changes to make it work. Will share it.

Topic		Replies	Views
Decoder generate with prompts of variable lengths? 🤗Transformers	0	664	May 25, 2022
Issue with Decoding in HuggingFace 🤗Tokenizers	2	3878	March 24, 2022
Batch generation with GPT2 🤗Transformers	12	17189	January 16, 2024
Parallelize Mistral/ llama2 output 🤗Transformers	1	154	May 25, 2024
Llama2 pad token for batched inference Models	7	15663	March 31, 2024