Inference Toolkit - Init and default template for custom inference

Amazon SageMaker
9

I created an example with a inference.py included in a model.tar.gz. You can find the whole repository here: GitHub - philschmid/sample-custom-inference-sagemaker-huggingface
You can find the inference.py here: sample-custom-inference-sagemaker-huggingface/inference.py at master · philschmid/sample-custom-inference-sagemaker-huggingface · GitHub

The structure of the archive is

code/
    inference.py
pytorch_model.bin
config.json
tokenizer.json
....
3 Likes