Is it necessary to create model in model.tar.gz format for deployment over amazon sagemaker

Amazon SageMaker
1

I am trying to deploy a custom data fine tune llam2 model over amazon sagemaker .
However the model compression is taking a lot more time , Just want to know is it possible to use an uncompressed model dir .

huggingface_model = HuggingFaceModel( 
    image_uri=get_huggingface_llm_image_uri("huggingface",version="0.8.2"),
    model_data="s3_path",  
    role=role,  
    transformers_version='4.28',  
    pytorch_version='2.0',  
    py_version='py310', 
    env=hub
)

can we use uncompress model dir for “s3_path” ?

2 Likes
2

We also have the same question, we can see from Sagemaker API this is allowed but not necessarily using the hugging face model. See Deploying uncompressed models - Amazon SageMaker

Does anyone know if the uncompressed approach can be used with a hugging face model deployment?