Deploying Open AI's whisper on Sagemaker

Amazon SageMaker
12

@thusken
According to this notebook, you should specify DataSerializer to serialize data.

hub = {
	'HF_MODEL_ID':'openai/whisper-base',
	'HF_TASK':'automatic-speech-recognition'
}


# create Hugging Face Model Class
huggingface_model = HuggingFaceModel(
    model_data = s3_location,
	transformers_version='4.17.0',
	pytorch_version='1.10.2',
	py_version='py38',
	env=hub,
	role=role, 
)

# deploy model to SageMaker Inference
audio_serializer = DataSerializer(content_type='audio/x-audio')
predictor = huggingface_model.deploy(
	initial_instance_count=1, # number of instances
	instance_type='ml.m5.xlarge', # ec2 instance type
    serializer=audio_serializer
)

Based on that issue, you can use serializers.

The Hugging Face inference toolkit supports all the transformers pipelines with their default inputs. The Toolkit implements several serializers to parse binary data, e.g., audio or images to the matching format for the transformers pipeline, e.g., PIL or np.

So, for inference below code worked in my environment.

audio_path = "sample1.flac"

res = predictor.predict(data=audio_path)
print(res)
2 Likes