New to Docker model getting stuck

Preface: I am kind of new to using docker.

I am currently creating an app that uses the colqwen2-v1.0 model from hugging face. I downloaded the model and saved it locally. When I run the fastapi app on my local computer everything works fine. I dockerized the app so that I can put in on an EC2 instance. The current probem is that whenever I try and run the API calls that would usually load the model and do something, the model just gets stuck loading the shards. Like nothing is happening. Is there something wrong with how I am using the model?

Any input is appreciated.

1 Like