Can't Deploy Hugging Face Endpoint on Azure

[Pbrezina]

Hello,
I’ve recently found out that there is a Hugging Face Endpoint available in Azure and I wanted to give it try. Unfortunately, I’ve hit a brick wall while attempting to deploy the service. I’m trying to create a model for Token Classification based on microsoft/layoutlmv2-base-uncased. I’ve also tried to create a model for Text Classification based on distilbert-base-uncased-finetuned-sst-2-english but without any success. My deployment configuration for Token Classification looks like this:

1. Basics:
   

   

2. Compute + Network:
   Instance Type: Standard_DS2_v2
   Min Instance Count: 1
   Max Instance Count: 2
   Scale Target Request Per Minute: 1000
   Use Azure Private Link: unchecked

3. Tags: I left those as is.

The validation of the settings is successful and I click the button to create the resource. However, the deployment fails with Conflict status after a few minutes (about 4-5 minutes) with the following raw error message:

{
  "code": "DeploymentFailed",
  "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.",
  "details": [
    {
      "code": "ApplianceDeploymentFailed",
      "message": "The operation to create appliance failed. Please check operations of deployment 'hugtokclass' under resource group '/subscriptions/f6b17c2b-afa1-437e-9b38-3c41d81dbac2/resourceGroups/mrg-huggingface-20220608092815'. Error message: 'At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.'"
    }
  ]
}

Unfortunately, this doesn’t tell me much as to what could be the problem. I am using Visual Studio Enterprise subscription to deploy the model. I am not sure what I am doing wrong here, whether I incorrectly filled in the configuration settings before deployment despite the validation being successful, or whether I simply can’t deploy the service using the Visual Studio Enterprise subscription. Any assistance with the HF Endpoint deployment would be appreciated as I am currently at a loss.

[philschmid]

Hello @Pbrezina,

Thank you for opening this thread and for providing feedback of the Azure Endpoints (Preview).
I am happy to hear that the deployment of the distilbert model worked.

For your LayoutLM test. You are trying to deploy a pre-trained model ( microsoft/layoutlmv2-base-uncased), which isn’t fined-tuned for any downstream task. Meaning that the task “token-classifcation” is not compatible with the model. Even if you would fine-tune/ use a fine-tuned layoutLM model would it fail since the token-classification task currently doesn’t support non NLP models, like LayoutLM

We are working on adding support for Vision and Multimodality (LayoutLM) with custom processing for GA of the service that you will then be able to use LayoutLM with your own code to run document information extraction.
In the meantime you could checkout this space on how this would work LayoutLMv 2 FUNSD - a Hugging Face Space by nielsr

[Pbrezina]

Hello @philschmid,

thank you for the response.

You misunderstood me regarding the distilbert model for text classification. The deployment of distilbert endpoint is also failing with the conflict status and the same error message as the deployment of LayoutLM. Now I understand that the LayoutLM model deployment won’t work since it is not finetuned and is not currently supported, but I am still confused why distilbert doesn’t work for me either.

[philschmid]

Could you please re-try it with distilbert and share the managed-application-rg name?

[Pbrezina]

Hello @philschmid,

This is my configuration when deploying the distilbert model endpoint.

I left the managed resource group to the default automatically generated value. My settings for compute+networking and tags are the same as before.

The deployment fails with Conflict status with following raw error message:

{
  "code": "DeploymentFailed",
  "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.",
  "details": [
    {
      "code": "ApplianceDeploymentFailed",
      "message": "The operation to create appliance failed. Please check operations of deployment 'hugtextclass' under resource group '/subscriptions/f6b17c2b-afa1-437e-9b38-3c41d81dbac2/resourceGroups/mrg-huggingface-20220609090948'. Error message: 'At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.'"
    }
  ]
}

I’ve also looked into deployment operation details and found a message regarding the Conflict status:

{
    "status": "Failed",
    "error": {
        "code": "ApplianceDeploymentFailed",
        "message": "The operation to create appliance failed. Please check operations of deployment 'hugtextclass' under resource group '/subscriptions/f6b17c2b-afa1-437e-9b38-3c41d81dbac2/resourceGroups/mrg-huggingface-20220609090948'. Error message: 'At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.'",
        "details": [
            {
                "code": "Conflict",
                "message": "{\r\n  \"status\": \"failed\",\r\n  \"error\": {\r\n    \"code\": \"ResourceDeploymentFailure\",\r\n    \"message\": \"The resource operation completed with terminal provisioning state 'failed'.\",\r\n    \"details\": [\r\n      {\r\n        \"code\": \"DeploymentScriptError\",\r\n        \"message\": \"The provided script failed with the following error:\\r\\nERROR: {\\\"status\\\":\\\"Failed\\\",\\\"error\\\":{\\\"code\\\":\\\"DeploymentFailed\\\",\\\"message\\\":\\\"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.\\\",\\\"details\\\":[{\\\"code\\\":\\\"Conflict\\\",\\\"message\\\":\\\"{\\\\r\\\\n \\\\\\\"status\\\\\\\": \\\\\\\"Failed\\\\\\\",\\\\r\\\\n \\\\\\\"error\\\\\\\": {\\\\r\\\\n \\\\\\\"code\\\\\\\": \\\\\\\"ResourceDeploymentFailure\\\\\\\",\\\\r\\\\n \\\\\\\"message\\\\\\\": \\\\\\\"The resource operation completed with terminal provisioning state 'Failed'.\\\\\\\",\\\\r\\\\n \\\\\\\"details\\\\\\\": [\\\\r\\\\n {\\\\r\\\\n \\\\\\\"message\\\\\\\": \\\\\\\"SubscriptionNotRegistered: Resource provider [N/A] isn't registered with Subscription [N/A]. Please see troubleshooting guide, available here: https://aka.ms/register-resource-provider\\\\\\\",\\\\r\\\\n \\\\\\\"details\\\\\\\": [],\\\\r\\\\n \\\\\\\"additionalInfo\\\\\\\": [\\\\r\\\\n {\\\\r\\\\n \\\\\\\"type\\\\\\\": \\\\\\\"MmsOperation\\\\\\\",\\\\r\\\\n \\\\\\\"info\\\\\\\": {\\\\r\\\\n \\\\\\\"id\\\\\\\": \\\\\\\"279fddd5-8685-4e69-b175-a4a7c027dc1b\\\\\\\",\\\\r\\\\n \\\\\\\"operationType\\\\\\\": \\\\\\\"OnlineEndpoint\\\\\\\",\\\\r\\\\n \\\\\\\"state\\\\\\\": \\\\\\\"Failed\\\\\\\",\\\\r\\\\n \\\\\\\"createdTime\\\\\\\": \\\\\\\"2022-06-09T07:16:09.8692253Z\\\\\\\",\\\\r\\\\n \\\\\\\"endTime\\\\\\\": \\\\\\\"2022-06-09T07:16:30.929291Z\\\\\\\",\\\\r\\\\n \\\\\\\"error\\\\\\\": {\\\\r\\\\n \\\\\\\"message\\\\\\\": \\\\\\\"SubscriptionNotRegistered: Resource provider [N/A] isn't registered with Subscription [N/A]. Please see troubleshooting guide, available here: https://aka.ms/register-resource-provider\\\\\\\"\\\\r\\\\n },\\\\r\\\\n \\\\\\\"resourceLocation\\\\\\\": \\\\\\\"/modelmanagement/v1.0/subscriptions/f6b17c2b-afa1-437e-9b38-3c41d81dbac2/resourceGroups/mrg-huggingface-20220609090948/providers/Microsoft.MachineLearningServices/workspaces/distilberttestkfealm5afkpq4/onlineEndpoints/distilberttestkfealm5afkep1\\\\\\\",\\\\r\\\\n \\\\\\\"operationLog\\\\\\\": \\\\\\\"\\\\\\\",\\\\r\\\\n \\\\\\\"parentRequestId\\\\\\\": \\\\\\\"635b702a-d27e-4408-b6b3-f589c57d66d6\\\\\\\"\\\\r\\\\n }\\\\r\\\\n }\\\\r\\\\n ]\\\\r\\\\n }\\\\r\\\\n ]\\\\r\\\\n }\\\\r\\\\n}\\\"}]}}. Please refer to https://aka.ms/DeploymentScriptsTroubleshoot for more deployment script information.\"\r\n      }\r\n    ]\r\n  }\r\n}"
            }
        ]
    }
}

What is interesting in this status message is the “Resource provider [N/A] isn’t registered with Subscription [N/A]”. However, I am not sure how to interpret it.

[philschmid]

@Pbrezina could you go on the azure portal to subscription → select your subscription → navigate to “resource providers” (left side) → search for “machine” → check if “Microsoft.MachineLearningServices” are “registered”

[Pbrezina]

@philschmid I checked the resource providers and I do have Microsoft.MachineLearningServices and Microsoft.MachineLearning registered.

[philschmid]

@Pbrezina we are heavily working with the Microsoft team to make the experience better. It is strange to see that it still doesn’t work and the error is saying N/A provider.

Could you check if Microsoft.Authorization is also registered?

Thank you for your help and testing the preview

[Pbrezina]

@philschmid yes, Microsoft.Authorization is registered as well.