Running batch transform in Sagemaker on a Huggingface model from the Hub with parameters

Beginners
1

Hi,

I am trying to run batch transform using facebook/bart-large-mnli on Sagemaker following the directions in this article: Deploy models to Amazon SageMaker

Is there a way to run it in a multi-label mode?
I see in this example its possible when creating a classifier, but I couldn’t find a way to do it with the batch transform sagemaker SDK

1 Like
2

Did you manage to run it. Im trying to run the exact same model in batch. So far the closest I’ve gotten is putting the parameters at each record in the json file like this and then putting each record in a line

{“inputs”: “pues claro que presenta domiciliacion, se cobraron de mi cuenta. mejor digame como recuperar ese dinero porque si no voy directo con profeco”, “parameters”: {“candidate_labels”: “tarjeta de credito,retiro,cajero,efectivo,transferencia,aclaracion,credito”, “multi_label”: false}}
{“inputs”: “pues claro que presenta domiciliacion, se cobraron de mi cuenta. mejor digame como recuperar ese dinero porque si no voy directo con profeco”, “parameters”: {“candidate_labels”: “tarjeta de credito,retiro,cajero,efectivo,transferencia,aclaracion,credito”, “multi_label”: false}}

but my processing job still fails with this error

1 Like
3

I have a similar issue. I could not use an inference.py overwrite a batch transform huggingface model from the hub. Nothing in my inference.py is printed in the log and it still gives me the error from the original function sagemaker-huggingface-inference-toolkit/handler_service.py at main · aws/sagemaker-huggingface-inference-toolkit · GitHub.

hub = {
    'HF_MODEL_ID':'facebook/bart-large-mnli', 
    'HF_TASK':'zero-shot-classification' 
}

huggingface_model = HuggingFaceModel(
    env=hub,
    role=role,
    entry_point="inference1.py",
    source_dir = "./code" ,
    transformers_version='4.17.0',
    pytorch_version='1.10.2',
    py_version='py38',
)

batch_job = huggingface_model.transformer(
    instance_count=1,
    instance_type='ml.m5.4xlarge',
    assemble_with = 'Line',
    accept = "application/json",
    max_payload =1, 
    output_path=output_s3_path, 
    strategy='SingleRecord'
)
batch_job.transform(
    data= s3_data_input, 
    split_type="Line",  
    content_type="application/json"
)```