How are the inputs tokenized when model deployment?

Amazon SageMaker
8

You can learn more about the Inference Toolkit here: Deploy models to Amazon SageMaker

In addition to the Hugging Face Inference Deep Learning Containers, we created a new Inference Toolkit for SageMaker. This new Inference Toolkit leverages the pipelines from the transformers library to allow zero-code deployments of models, without requiring any code for pre- or post-processing.

Meaning the parameters key supports all optional parameters of the transformers pipelines: Pipelines

1 Like