Hi All, Just want to know, is there any way to batch decode variable length sentences. For example [S1, S2] , where S1 has 5 words abd S2 has 10 words . Can we decode it using GPT2 , BERT etc?

Variable length batch decoding

s4sarath October 5, 2020, 11:53am 9

No problem @valhalla . Appreciate your response.
I have implemented this feature locally.

Off topic, would like to know what is your thoughts on this.

Topic		Replies	Views
Decoder generate with prompts of variable lengths? 🤗Transformers	0	543	May 25, 2022
Training with varying lengths of sequences Beginners	0	927	May 31, 2023
Issue with Decoding in HuggingFace 🤗Tokenizers	2	2346	March 24, 2022
Results of model.generate are different for different batch sizes of the decode-only model Beginners	6	3095	April 14, 2024
T5 decoder predicting tokens even after hitting end of sequence token, i.e </s> 🤗Transformers	4	189	February 26, 2024