Creating inference endpoint with custom handler - is this how it should work?

I don’t seem to quite understand how to deploy a model with a custom handler as an inference endpoint. Specifically, I’m trying to use this model (which alread has a custom handler): philschmid/stable-diffusion-v1-4-endpoints · Hugging Face

I went through these steps trying to follow the tutorial:

  1. Initialize Git LFS: git lfs install
  2. Clone the desired repo: git clone
  3. Install huggingface-hub python package (in a venv): python -m venv venv + source venv/bin/activate + pip install huggingface-hub
  4. Login to Huggingface: huggingface-cli login → Choose Yes when asked to add token as git credential
  5. Run this command (don’t really understand what it does): git config --global credential.helper store

As far as I understand, I don’t need to change anything in the code, since it already has a custom handler. I now need to push the whole thing to a repo that I created before. It’s currently empty: s

  1. I move up one folder into /stable-diffusion-v1-4-endpoints
  2. I delete the existing .git so I can push the whole code to my own repo: sudo rm -r .git
  3. I initialize a new git: git init + git remote add origin master + git remote set-url origin
  4. Add all files to commit to my new remote repo: git add * + git commit -m "my first commit" + git push origin master

At this point I’m getting a git error error failed to push some refs to

As far as I understand this should be the right way. Once I have code with a in my repo, I should be able to find it in the inference endpoints search mask, right? (see screenshot)

Is this the right way to do it? And if so: Maybe someone could help me fix the Git error? That would be great!

Anybody? If someone has a link/resource that could help me solve this by myself, I’d be more than happy! :slight_smile:

cc @philschmid

btw, might be better to post this in the “inference endpoints” section, rather than “beginners”

1 Like

Hello @eykarim,

You can check out this tutorial: Custom Inference with Hugging Face Inference Endpoints
or the documentation on how to create custom handlers: Create custom Inference Handler

Those include e2e example with uploading the model.

FYI we also have a dedicated inference endpoints section in the forum.

1 Like

Hi Phil! Thanks for the tutorial. I worked through it a second time yesterday and now have a model deployed: eykarim/stable-diffusion-v1 · Hugging Face

As I wrote in my original post, I was assuming that once I have a model deployed with a custom handler (marked as “endpoints-template”) I should be able to deploy it with an Inference Endpoint. However, the model does not show up in the search bar, nor do many of the other models. I don’t understand what requirements a model needs to fullfil in order to be deployed as an Inference Endpoint. Why do some models with an endpoint handler show up in the search bar and others don’t?

I responded in your other thread: Why is my model not showing up in the search bar for Inference Endpoints? - #2 by philschmid

1 Like