Fine tuned phi2 model loses context once loaded from local

Hi Awesome people,

Currently I am exploring the phi-2 fine tuning and I am at the phase where results it yielded is pretty good. However, I am facing an issue when I try to load and run the model for inference. It stared giving hallucinations and non-relevant answers. Some times similar questions as the answers to the question prompted.

I have saved the model after finetuning (using PEFT) with pipeline and also with SFTTrainer
. i.e.
(at separate instances)



and tried to load them via


pipe = pipeline(task="text-generation", model='./path_to_local_dir', tokenizer=tokenizer, max_length=250, trust_remote_code=True)

but in both cases, my finetuned LLM (phi-2) is behaving abnormally. Could anyone please suggest where I have gone wrong ? Every help is appreciated.