I am a littile confused. In transformer based LLM inference. We always set the input length as the max_input length and padding 0 , or we dynamically get the actual input length and decrease the computation?

Inference input token number set as the max length always?

vivek9840 April 21, 2024, 8:28pm 11

these are the sources which i learned from. might help you.

Topic		Replies	Views
Sequences shorter than model's input window size 🤗Transformers	2	1163	January 4, 2022
Why does padding = 'max_length' cause much slower model inference? Models	1	615	June 8, 2023
How to set 'max_length' properly when using pipeline? 🤗Transformers	4	1342	November 18, 2024
[Tokenizers]What this max_length number? Beginners	3	2412	March 3, 2025
Confused about max_length and max_new_tokens 🤗Transformers	7	35487	September 5, 2024