Containerizing Huggingface Transformers for GPU inference with Docker and FastAPI

Hi everyone!

A while ago I was searching on the HF forum and web to create a GPU docker and deploy it on cloud services like AWS.
Couldn’t find a comprehensive guide that showed how to create and deploy transformers on GPU.

So decided to do one myself and publish it so that it is helpful for others who want to create a GPU docker with HF transformers and deploy it.

Just wanted to add the resource here.

Github: GitHub - ramsrigouthamg/GPU_Docker_Deployment_HuggingFace_Summarization: Huggingface inference with GPU Docker on AWS
Detailed Youtube Video: Containerizing Huggingface Transformers for GPU inference with Docker and FastAPI on AWS - YouTube

Happy learning!

Cheers,
Ramsri

3 Likes