BERT MLM finetuning with custom embeddings

Hi, I have some embeddings that I would like to finetune using BERT mlm script. Its an embedding.pt file. While passing this embedding with --model_name_or_path to run_mlm.py, it throws an error saying pytorch_model.bin not found in the folder. Is this only about converting .pt to .bin file or is there some other way? I also tried setting from_pt=True in the script while loading the model using AutoModelforMaskedLM. Any suggestions?