Dynamically change max_new_tokens in a pipeline

Beginners
1

Is there anyway to change the max_new_tokens used in a pipeline without having to load the whole pipeline again?

I am using gemma27b-it and just playing around with the model, starting with a base prompt and then stringing the model’s answers with new prompts and so on. The only thing is that right now the max_new_tokens variable is static, and I want to control it every time I call the pipeline object such that for a question like ‘Paris is the capital of France: T or F?’ I want to set max_new_tokens = 1 and for more elaborate questions, to raise this limit. Is this possible?

2

Specify max_new_tokens when you call the pipeline.

Ex:
Q = “Paris is the capital of France: T or F?”
T_or_F = pipeline(Q, max_new_tokens=1)
Elaborated = pipeline(f“Elaborate the response to ‘{Q}’: {T_or_F}.”, max_new_tokens=1024)