How to add a stop sequence to a Pipeline?

🤗Transformers
1

Hello,

I know I can do this with model.generate but I would like to know if it is possible to add an arg for an stop sequence with the Pipeline.

For example:

model = AutoModelForCausalLM.from_pretrained(
“microsoft/Phi-3-mini-128k-instruct”,
device_map=“cuda”,
torch_dtype=“auto”,
trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained(“microsoft/Phi-3-mini-128k-instruct”)

pipe = pipeline(
“text-generation”,
model=model,
tokenizer=tokenizer,
max_new_tokens=1000,
return_full_text=False,
temperature=0.1,
stop=[“my stop chars”]??? Here add my stop sequence argument which I am struggling to find how to do it
)

1 Like