Hi, I am getting different sequence for microsoft/Phi-3-mini-4k-instruct when i set return_dict_in_generate=True in model.generate? Example output: [ {'role': '…

[Screen Shot 2024-06-09 at 6.36.06 PM] Confirming that following change fixes the issue. The key change is adding eos_token_id

Difference in return sequence for Phi3 model

exploiter345 June 9, 2024, 11:28pm 7

The Flash Attention warning is observed in both cases, but let me see if i can avoid it

Topic		Replies	Views
Generate() returns full prompt plus answer 🤗Transformers	1	6272	February 19, 2024
Model.generate generates same output for different inputs 🤗Transformers	1	619	November 13, 2023
Attention_mask missing from generate() output 🤗Transformers	0	197	November 16, 2023
Pipeline vs model.generate() Beginners	11	14119	July 16, 2025
Always output generation config in terminal Models	1	252	November 15, 2023