I prompt-tuned an adapter for LLaMA 7B and saved it to S3 after training without merging it to the base model first (i.e., I only have the adapter saved in S3). I have not pushed it up to the hub yet. I want to deploy a model using this adapter on SageMaker using HuggingFaceModel, but I’m not sure how to do this. Would I need to merge it to the base model separately first, or is there a way to merge it within HuggingFaceModel?
I was thinking I might be able to write a script that merges the adapter to the model and pass that in through entry_point. Would this work? How would I go about writing such a script?