How are the inputs tokenized when model deployment?

Amazon SageMaker
9

Hi again…

I tried the solution on endpoints and it works perfectly, … but I’m having issues with batch transform prediction (this is actually my goal).

The batch transform method requires the data to be predicted be in an s3 jsonl file. So I created a python dictionary like this and save later into jsonl on S3.
python file

[
 {"inputs": "long sentence 1", "parameters": {"trucation": True}},  
 {"inputs": "long sentence 2", "parameters": {"trucation": True}}, 
 {"inputs": "long sentence 3", "parameters": {"trucation": True}}, 
 {"inputs": "long sentence 4", "parameters": {"trucation": True}},
]

saved on:

test.jsonl on S3

{"inputs": "long sentence 1", "parameters": {"trucation": true}} 
{"inputs": "long sentence 2", "parameters": {"trucation": true}}
{"inputs": "long sentence 3", "parameters": {"trucation": true}}
{"inputs": "long sentence 4", "parameters": {"trucation": true}}

be aware that in jsonl “True” becomes “true” and maybe this is the cause of the error?

Then this is my code:


# create Hugging Face Model Class
huggingface_model = HuggingFaceModel(
   model_data=model_path,  # path to your trained sagemaker model
   role=role, # iam role with permissions to create an Endpoint
   transformers_version="4.6", # transformers version used
   pytorch_version="1.7", # pytorch version used
   py_version="py36", # python version of the DLC
)

# create Transformer to run our batch job
batch_job = huggingface_model.transformer(
    instance_count=1,
    instance_type='ml.p3.2xlarge',
    output_path=upload_path, # we are using the same s3 path to save the output with the input
    strategy='SingleRecord')

# starts batch transform job and uses s3 data as input
batch_job.transform(
    data=s3_file_uri,
    content_type='application/json',    
    split_type='Line')

And I have this error:

image
image857×596 85.7 KB

I don’t know what can I do for makes this code work…