Custom Attributes for Asynchronous Endpoint Sagemaker

Amazon SageMaker
1

I am using AWS Sagemaker to host an asynchronous endpoint.
I have a custom inference.py code for this task.
This is how i invoke the endpoint using boto3:

response = client.invoke_endpoint_async(
    EndpointName='string',
    ContentType='string',
    Accept='string',
    CustomAttributes='string',
    InferenceId='string',
    InputLocation='string',
    RequestTTLSeconds=123,
    InvocationTimeoutSeconds=123
)

I use https://github.com/aws/sagemaker-huggingface-inference-toolkit to custom the endpoint.
However, I have a question that: where can I access the CustomAttributes. in sagemaker-huggingface-inference-toolkit?

Any help is appreciated. Thanks

1 Like
2

Hello, I assume you refer to these environment variables? GitHub - aws/sagemaker-huggingface-inference-toolkit

It should be added in the “hub” dictionnary you pass to your Model object used to create the endpoint.

1 Like