Missing likelihood for all the classes in inference after BERT model deployed in SageMaker

Beginners
1

Hi,

I have recently deployed the FinBERT model in SageMaker, and I am using it for inference in a text classification task. When I call the predictor.predict(data) I get only the label and probability of the predicted one (the one with the maximum value), but I am missing the ones for the other classes. I have already tried passing the input data to the predict method with the inputs and parameters but this doesn’t change anything. How can I retrieve all the labels and their probs when doing the inference on a Hugginface model in SageMaker?

I’m also posting the full code here if this helps in any way:

Model Deployment

from sagemaker.huggingface import HuggingFaceModel
import sagemaker

role = sagemaker.get_execution_role()
# Hub Model configuration. https://huggingface.co/models
hub = {
    'HF_MODEL_ID':'ProsusAI/finbert',
    'HF_TASK':'text-classification'
}

# create Hugging Face Model Class
huggingface_model = HuggingFaceModel(
    transformers_version='4.6.1',
    pytorch_version='1.7.1',
    py_version='py36',
    env=hub,
    role=role, 
)

# deploy model to SageMaker Inference
predictor = huggingface_model.deploy(
    initial_instance_count=1, # number of instances
    instance_type='ml.m5.xlarge' # ec2 instance type
)

Get Predictor

predictor = sagemaker.predictor.Predictor(
    endpoint_name = 'huggingface-pytorch-inference-endpoint',
    sagemaker_session = sess,
    serializer = sagemaker.serializers.JSONSerializer()
    
)

Inference

import json

data = {
    'inputs': ['LYXe partnering with Fan Zone a recent investment and partner of Porsche is a true signal of a continued growth in its e'], 
    'parameters': {
        'candidate_labels': [
            'negative', 
            'positive', 
            'neutral'
        ]
    }
}
print(json.loads(predictor.predict(data)))

Result

[{'label': 'positive', 'score': 0.9388243556022644}]

What I am expecting is something like [{'label': 'negative', 'score': ....}, {'label': 'neutral', 'score': ....}, {'label': 'positive', 'score': ....}]

Thank you!

2

Did you find the solution?