Protecting a private model when using inference endpoints

Hello all,

We recently started an inference endpoint for our model. This endpoint will be used by clients using a local application. It is, however, unclear to us if using the organization API token the customer has read access to our model. Could the customer theoretically download our model using this token? Or can the organization token only be used for inference?

Hello @RaphaelK,

You could create a user account and add him to your organization, with read permissions and then generate a UAT (User Access Token) for your customer.
In inference Endpoints you don’t have to use the org token for API calls to your endpoints, you can use tokens of users who are part of the organization.

Hi Phil,

Thank you for your reply. If I am correct the model should then be located on my personal account right, not on the organization?

Not particularly. You can host the model in the organization, but use a personal account (token) to access it if the account is part of the organization.

I just tried this and, using the customer account with a read UAT, I could clone the model git repository. This is what we would like to prevent.

What do you mean by clone? Cloning the repository deployed to the endpoint? Thats not possible. A token who can request the endpoint can also read the repository. You could hide the information which repository is used.

Thank you that answers my question. We will hide the repository information then.