How to set 'max_length' properly when using pipeline?

🤗Transformers
5

@jiaweihuang

prompt = 'What is the answer of 1 + 1?'
pipe = pipeline(
            "text-generation",
            tokenizer=tokenizer,
            model=model,
            do_sample=True,
            truncation=True,
            padding='max_length',
            num_return_sequences=2,
            temperature=1.0,
            num_beams=1,
            max_length=1024,
            max_new_tokens=512,
        )
messages = [
    {"role": "user", "content": prompt},
]
-ret = pipe(messages)
+ret = pipe(messages, max_length= 1024)

also pretty sure all of these prameters are used in the generation and not in the initialization

1 Like