Load local model with WhisperModel

I am trying to load a local model but get the error OSError: Incorrect path_or_model_id
# save model locally
model = WhisperModel.from_pretrained("openai/whisper-tiny")
PreTrainedModel.save_pretrained(model, <my_path>

Error when doing following:
model = WhisperModel.from_pretrained(pretrained_model_name_or_path=<my_path>, local_files_only=True)