Could not load or save IA3 fine tuned model over Roberta properly

Beginners
1

I have tried saving the fined tuned model by

  1. trainer.save_model(“./PEFT_model”)
  2. trainer.model._save_pretrained(“./peft_sv_2model”)

and loading it using
1.base_model = AutoModelForSequenceClassification.from_pretrained(“roberta-large”,
problem_type=“multi_label_classification”,
num_labels=len(labels),
id2label=id2label,
label2id=label2id)
Peft_Model = PeftModel.from_pretrained(base_model, “peft_sv_2model”)
Peft_Model.merge_and_unload()

Loading trainer model directly after creating config file manually

  1. Peft_Model = AutoModelForSequenceClassification.from_pretrained(“./PEFT_model”)

Both the ways are not giving same results as it was giving with the fine tuned trainer just after the training. The performance difference is huge (above 5-10%). Any suggestions of what I am doing wrong here.