Calling Image Classification Model Deployed in SageMaker Endpoint

TBRE · February 21, 2023, 4:19pm

Hi Community!

I am deploying this HF model (EdBianchi/vit-fire-detection · Hugging Face) on AWS SageMaker.

Once deployed, I have an S3 bucket where I am uploading .jpg images too. This S3 bucket triggers a Lambda function to invoke the SageMaker endpoint.

However, I am running into difficulty getting this endpoint working! Playing around, I have seen the endpoint can handle a maximum file size of 5 MB, which I have accounted for.

I have tried calling the endpoint with the image/content type set to ‘image/jpeg’ and passing the file as is from S3. I have tried converting the image file to bytes also. Nothing I have attempted is allowing me to successfully call the endpoint and return the classification.

Any help in this regard would be greatly appreciated!

marshmellow77 · February 21, 2023, 8:22pm

Hi Ross, what’s the error that you’re getting when trying to invoke the endpoint?

TBRE · February 21, 2023, 8:34pm

Hi!

Here is the error currently:

“message”: “cannot identify image file \u003c_io.BytesIO object at 0x7f3d78578810\u003e”

marshmellow77 · February 21, 2023, 8:46pm

Hm, ok, I probably need some more info. What transformers version are you using, what does your inference script look like, how do you invoke the endpoint, what serializer are you using, etc. Anything than can help

TBRE · February 21, 2023, 8:52pm

So I am using the example Deployment Code on HuggingFace.

from sagemaker.huggingface import HuggingFaceModel
import sagemaker

role = sagemaker.get_execution_role()
# Hub Model configuration. https://huggingface.co/models
hub = {
	'HF_MODEL_ID':'EdBianchi/vit-fire-detection',
	'HF_TASK':'image-classification'
}

# create Hugging Face Model Class
huggingface_model = HuggingFaceModel(
	transformers_version='4.17.0',
	pytorch_version='1.10.2',
	py_version='py38',
	env=hub,
	role=role, 
)

# deploy model to SageMaker Inference
predictor = huggingface_model.deploy(
	initial_instance_count=1, # number of instances
	instance_type='ml.m5.xlarge' # ec2 instance type
)
})

For the Lambda Function I have a trigger attached to S3 when the JPEG File lands and here is the code to call the SageMaker endpoint:

import os
import io
import boto3
import json
import base64

# grab environment variables
ENDPOINT_NAME = os.environ['ENDPOINT_NAME']
runtime= boto3.client('runtime.sagemaker')

# output variables
output_bucket = 'output-wildfire-images'

def lambda_handler(event, context):
    try:
        # Get the object from the S3 event
        s3 = boto3.client("s3")
        bucket = event['Records'][0]['s3']['bucket']['name']
        key = event['Records'][0]['s3']['object']['key']
        
        # Download the image file from S3
        response = s3.get_object(Bucket=bucket, Key=key)
        body = response['Body'].read()
        
        # Encode the image as a base64-encoded string
        img_b64 = base64.b64encode(body).decode('utf-8')
        
        # Convert the base64-encoded string back to bytes
        img_bytes = bytes(img_b64, 'utf-8')
        
        # Invoke the SageMaker endpoint
        response = runtime.invoke_endpoint(
            EndpointName=ENDPOINT_NAME,
            Body=img_bytes,
            ContentType='image/jpeg')
        print(response)

        # Parse the JSON response from the SageMaker endpoint
        result = json.loads(response['Body'].read().decode())
        print(result)
        
        # Upload the result to the output S3 bucket
        output_key = key.split('.')[0] + '.json'
        s3_client = boto3.client('s3')
        s3_client.put_object(Body=json.dumps(result), Bucket=output_bucket, Key=output_key)

        return {
            'statusCode': 200,
            'body': result
        }
    except Exception as e:
        print('Exception:', str(e))
        return {
            'statusCode': 500,
            'body': str(e)
        }

I am not sure if the .jpg file needs to be converted to bytes for the Hugging Face model to make the prediction on it…

Any help would be appreciated

marshmellow77 · February 22, 2023, 4:04am

Ah ok, that is helpful. I believe the problem is that Sagemaker predictor uses (I believe) the JSON serializer when not specified otherwise.

Have a look at this sample notebook that also handles images: notebooks/sagemaker-notebook.ipynb at main · huggingface/notebooks · GitHub

You can see that this notebook uses a DataSerializer instead:

# create a serializer for the data
image_serializer = DataSerializer(content_type='image/x-image') # using x-image to support multiple image formats

# deploy model to SageMaker Inference
predictor = huggingface_model.deploy(
	initial_instance_count=1, # number of instances
	instance_type='ml.g4dn.xlarge', # ec2 instance type
    serializer=image_serializer, # serializer for image data.
)

Hopefully that fixes your issue. You can learn more about Sagemaker Serializers here

Cheers
Heiko

TBRE · March 3, 2023, 7:30pm

Hi, I am still having issues getting this wildifre model endpoint to call correctly and return the result.

I have worked with the example you sent for image segmentation, which works fine - notebooks/sagemaker-notebook.ipynb at main · huggingface/notebooks · GitHub.

Here is the code for deploying the wildifre model in SageMaker Notebook that I am using.

import base64
import json
import sagemaker

# Role
role = sagemaker.get_execution_role()
print(role)

# SageMaker 
from sagemaker.huggingface import HuggingFaceModel
from sagemaker.serializers import DataSerializer

role = sagemaker.get_execution_role()
# Hub Model configuration. https://huggingface.co/models
hub = {
	'HF_MODEL_ID':'EdBianchi/vit-fire-detection',
	'HF_TASK':'image-classification'
}

# create Hugging Face Model Class
huggingface_model = HuggingFaceModel(
    transformers_version='4.17.0',
	pytorch_version='1.10.2',
	py_version='py38',
	env=hub,
	role=role, 
)

# create a serializer for the data
image_serializer = DataSerializer(content_type='image/x-image') # using x-image to support multiple image formats

# deploy model to SageMaker Inference
predictor = huggingface_model.deploy(
	initial_instance_count=1, # number of instances
	instance_type='ml.m5.xlarge', # ec2 instance type
        serializer=image_serializer, # serializer for image data.
)

# download the image 
!wget https://huggingface.co/datasets/hf-internal-testing/fixtures_ade20k/raw/main/ADE_val_00000001.jpg

# File path
image_path = "ADE_val_00000001.jpg"

# Call the endpoint
with open(image_path, "rb") as data_file:
  image_data = data_file.read()
  res = predictor.predict(data=image_data)
  print(res)

I run into this error…

ModelError: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received client error (400) from primary with message "{
  "code": 400,
  "type": "InternalServerException",
  "message": "an integer is required (got type str)"
}

philschmid · March 6, 2023, 8:45am

Hello @TBRE,

can you share the logs from cloudwatch?

TBRE · March 6, 2023, 8:47pm

Hi @philschmid,

Here is the CloudWatch log response.

"message": "cannot identify image file \u003c_io.BytesIO object at 0x7f3d783d4a40\u003e"

2023-02-21T15:46:24,565 [INFO ] W-EdBianchi__vit-fire-detec-4-stdout com.amazonaws.ml.mms.wlm.WorkerLifeCycle - mms.service.PredictionException: an integer is required (got type str) : 400

philschmid · March 6, 2023, 8:59pm

It seems that the container couldn’t parse your image correctly. Could you try as mime-type image/jpeg instead of x-image ?

TBRE · March 8, 2023, 2:26am

Tried changing to image/jpeg and still not working

philschmid · March 8, 2023, 7:59am

can you try a different image? it seems something is off with the request payload

Neesh · March 22, 2023, 11:59am

I’m having the same issue - I’ve tried appending “data:image/jpeg;base64,” and just sending the raw B64 image - however no approach works - did you find a way around this?

TBRE · March 23, 2023, 1:03am

Hi @Neesh,

In short, no I wasn’t able to.

I had a change in priorities as this was taking too long to set up.

I tried similar techniques to you and was not able to get the inference working.

Nakul24 · April 13, 2023, 7:27pm

I am having a similar issue. was anyone able to fix this?

TBRE · April 13, 2023, 9:39pm

Not yet! Can anyone from hugging face directly test this model and share some insight?

philschmid · April 14, 2023, 6:54am

Have you checked out this example? notebooks/sagemaker-notebook.ipynb at main · huggingface/notebooks · GitHub
It is using image-segementation but the process should be the same and it is tested and works.

Jamss · July 11, 2023, 1:59pm

I am also having issues setting up on inference api. Im using stablediffusion and follow phillipps examples.

Keep running into this cannot identify image file <_io.BytesIO error. The handler.py file works perfectly in colab.

Faiz4work · April 2, 2024, 6:18pm

try with “r” instead of “rb”

KeelyPowers · April 4, 2024, 11:06pm

Make sure your image not only meets the maximum file size, but is also encoded correctly. Make sure you are using the correct image format when calling the SageMaker endpoint. It’s also worth checking your S3 permissions and Lambda settings to make sure they’re configured correctly for file access.

Topic		Replies	Views
Endpoint Deployment Amazon SageMaker	1	1095	September 20, 2021
Can text-to-image models be deployed to a SageMaker endpoint? Amazon SageMaker	1	1947	July 8, 2022
Calling Sagemaker Endpoint for fine-tuned summarization model Amazon SageMaker	15	4942	March 22, 2024
Need help with hugging face API endpoint. ModelError: code "400" Beginners	0	303	March 3, 2024
Training model file too large and fail to deploy Amazon SageMaker	3	1368	July 3, 2023

Calling Image Classification Model Deployed in SageMaker Endpoint

Related topics