Cannot Deploy My Private Model for Inference Endpoint

[weiqis]

Hi,

I have a private model that I’d like to deploy in Inference Endpoint. But I cannot find it from the model search box. Is it impossible to deploy a private model or do I miss anything?

I already turned on inference in the model metadata:

license: creativeml-openrail-m
language:
  - en
pipeline_tag: text-generation
tags:
  - endpoints
  - text-generation-inference
inference: true

Any suggestions will be appreciated. Thanks.

[philschmid]

Hello @weiqis,

Private models can be deployed in the account that “owns” the repository. Meaning if you use an organization on inference endpoints, the model needs to be part of the organization.

Also to mention we had a bug until this afternoon where private repos where showing anymore but that’s fixed.

[weiqis]

Hi @philschmid , thanks for your quick reply. But the model is under my own account, and I’m not currently in any organization yet. My inference endpoint is also under my own account. So I have the full ownership of the private model. I just checked and I’m still not able to see my models.

[philschmid]

Can you share a screenshot of it?

[weiqis]

Absolutely. Here is my model (I don’t have any other information other than the metadata in my model card):

And here is the inference engine where I tried to search my model:

[beurkinger]

Hi @weiqis ! The issue should now be fixed. Please tell us if you are now able to import your private models.

Best,
Thibault

[weiqis]

Yep, I can see all my private models now. Thanks!