Hey, Had some quick questions regarding the Inference Toolkit. Is there a way to add an init function in the custom inference.py script. I was thinking I could just add what I needed in the model_fn function but when I tried running just the basics, I got an error attached below. This leads into th…

You can add a requirements.txt into the code/ and the archive and upload it to s3 and provide it as model_data. This should work. You can use my example to test it. Additionally, does the FrameworkModel class have the attribute dependencies, but it looks way more complex to add your dependencies. …

Inference Toolkit - Init and default template for custom inference

Amazon SageMaker

philschmid October 4, 2021, 7:09am 7

With deploy.py I meant the script which is creating your endpoint. This could also be a notebook or so.

Topic		Replies	Views
Loading inference.py separately from model.tar.gz Amazon SageMaker	4	1862	June 5, 2023
Help for inference.py code Amazon SageMaker	10	4003	March 8, 2022
HuggingFaceModel ignores code directory Amazon SageMaker	2	16	June 17, 2025
Install custom python libraries in HuggingFace DLC Amazon SageMaker	9	2003	August 8, 2022
Inference Toolkit - custom inference with multiple models Amazon SageMaker	1	635	April 4, 2024

Inference Toolkit - Init and default template for custom inference

Related topics