Can't Deploy Hugging Face Endpoint on Azure

Pbrezina · June 8, 2022, 8:02am

Hello,
I’ve recently found out that there is a Hugging Face Endpoint available in Azure and I wanted to give it try. Unfortunately, I’ve hit a brick wall while attempting to deploy the service. I’m trying to create a model for Token Classification based on microsoft/layoutlmv2-base-uncased. I’ve also tried to create a model for Text Classification based on distilbert-base-uncased-finetuned-sst-2-english but without any success. My deployment configuration for Token Classification looks like this:

Basics:

huggingface_res_dep_1715×813 27.3 KB
Compute + Network:
Instance Type: Standard_DS2_v2
Min Instance Count: 1
Max Instance Count: 2
Scale Target Request Per Minute: 1000
Use Azure Private Link: unchecked
Tags: I left those as is.

The validation of the settings is successful and I click the button to create the resource. However, the deployment fails with Conflict status after a few minutes (about 4-5 minutes) with the following raw error message:

{
  "code": "DeploymentFailed",
  "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.",
  "details": [
    {
      "code": "ApplianceDeploymentFailed",
      "message": "The operation to create appliance failed. Please check operations of deployment 'hugtokclass' under resource group '/subscriptions/f6b17c2b-afa1-437e-9b38-3c41d81dbac2/resourceGroups/mrg-huggingface-20220608092815'. Error message: 'At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.'"
    }
  ]
}

Unfortunately, this doesn’t tell me much as to what could be the problem. I am using Visual Studio Enterprise subscription to deploy the model. I am not sure what I am doing wrong here, whether I incorrectly filled in the configuration settings before deployment despite the validation being successful, or whether I simply can’t deploy the service using the Visual Studio Enterprise subscription. Any assistance with the HF Endpoint deployment would be appreciated as I am currently at a loss.

philschmid · June 8, 2022, 11:46am

Hello @Pbrezina,

Thank you for opening this thread and for providing feedback of the Azure Endpoints (Preview).
I am happy to hear that the deployment of the distilbert model worked.

For your LayoutLM test. You are trying to deploy a pre-trained model ( microsoft/layoutlmv2-base-uncased), which isn’t fined-tuned for any downstream task. Meaning that the task “token-classifcation” is not compatible with the model. Even if you would fine-tune/ use a fine-tuned layoutLM model would it fail since the token-classification task currently doesn’t support non NLP models, like LayoutLM

We are working on adding support for Vision and Multimodality (LayoutLM) with custom processing for GA of the service that you will then be able to use LayoutLM with your own code to run document information extraction.
In the meantime you could checkout this space on how this would work LayoutLMv 2 FUNSD - a Hugging Face Space by nielsr

Pbrezina · June 8, 2022, 12:18pm

Hello @philschmid,

thank you for the response.

You misunderstood me regarding the distilbert model for text classification. The deployment of distilbert endpoint is also failing with the conflict status and the same error message as the deployment of LayoutLM. Now I understand that the LayoutLM model deployment won’t work since it is not finetuned and is not currently supported, but I am still confused why distilbert doesn’t work for me either.

philschmid · June 8, 2022, 1:42pm

Could you please re-try it with distilbert and share the managed-application-rg name?

Pbrezina · June 9, 2022, 7:21am

Hello @philschmid,

This is my configuration when deploying the distilbert model endpoint.

I left the managed resource group to the default automatically generated value. My settings for compute+networking and tags are the same as before.

The deployment fails with Conflict status with following raw error message:

{
  "code": "DeploymentFailed",
  "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.",
  "details": [
    {
      "code": "ApplianceDeploymentFailed",
      "message": "The operation to create appliance failed. Please check operations of deployment 'hugtextclass' under resource group '/subscriptions/f6b17c2b-afa1-437e-9b38-3c41d81dbac2/resourceGroups/mrg-huggingface-20220609090948'. Error message: 'At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.'"
    }
  ]
}

I’ve also looked into deployment operation details and found a message regarding the Conflict status:

{
    "status": "Failed",
    "error": {
        "code": "ApplianceDeploymentFailed",
        "message": "The operation to create appliance failed. Please check operations of deployment 'hugtextclass' under resource group '/subscriptions/f6b17c2b-afa1-437e-9b38-3c41d81dbac2/resourceGroups/mrg-huggingface-20220609090948'. Error message: 'At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.'",
        "details": [
            {
                "code": "Conflict",
                "message": "{\r\n  \"status\": \"failed\",\r\n  \"error\": {\r\n    \"code\": \"ResourceDeploymentFailure\",\r\n    \"message\": \"The resource operation completed with terminal provisioning state 'failed'.\",\r\n    \"details\": [\r\n      {\r\n        \"code\": \"DeploymentScriptError\",\r\n        \"message\": \"The provided script failed with the following error:\\r\\nERROR: {\\\"status\\\":\\\"Failed\\\",\\\"error\\\":{\\\"code\\\":\\\"DeploymentFailed\\\",\\\"message\\\":\\\"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.\\\",\\\"details\\\":[{\\\"code\\\":\\\"Conflict\\\",\\\"message\\\":\\\"{\\\\r\\\\n \\\\\\\"status\\\\\\\": \\\\\\\"Failed\\\\\\\",\\\\r\\\\n \\\\\\\"error\\\\\\\": {\\\\r\\\\n \\\\\\\"code\\\\\\\": \\\\\\\"ResourceDeploymentFailure\\\\\\\",\\\\r\\\\n \\\\\\\"message\\\\\\\": \\\\\\\"The resource operation completed with terminal provisioning state 'Failed'.\\\\\\\",\\\\r\\\\n \\\\\\\"details\\\\\\\": [\\\\r\\\\n {\\\\r\\\\n \\\\\\\"message\\\\\\\": \\\\\\\"SubscriptionNotRegistered: Resource provider [N/A] isn't registered with Subscription [N/A]. Please see troubleshooting guide, available here: https://aka.ms/register-resource-provider\\\\\\\",\\\\r\\\\n \\\\\\\"details\\\\\\\": [],\\\\r\\\\n \\\\\\\"additionalInfo\\\\\\\": [\\\\r\\\\n {\\\\r\\\\n \\\\\\\"type\\\\\\\": \\\\\\\"MmsOperation\\\\\\\",\\\\r\\\\n \\\\\\\"info\\\\\\\": {\\\\r\\\\n \\\\\\\"id\\\\\\\": \\\\\\\"279fddd5-8685-4e69-b175-a4a7c027dc1b\\\\\\\",\\\\r\\\\n \\\\\\\"operationType\\\\\\\": \\\\\\\"OnlineEndpoint\\\\\\\",\\\\r\\\\n \\\\\\\"state\\\\\\\": \\\\\\\"Failed\\\\\\\",\\\\r\\\\n \\\\\\\"createdTime\\\\\\\": \\\\\\\"2022-06-09T07:16:09.8692253Z\\\\\\\",\\\\r\\\\n \\\\\\\"endTime\\\\\\\": \\\\\\\"2022-06-09T07:16:30.929291Z\\\\\\\",\\\\r\\\\n \\\\\\\"error\\\\\\\": {\\\\r\\\\n \\\\\\\"message\\\\\\\": \\\\\\\"SubscriptionNotRegistered: Resource provider [N/A] isn't registered with Subscription [N/A]. Please see troubleshooting guide, available here: https://aka.ms/register-resource-provider\\\\\\\"\\\\r\\\\n },\\\\r\\\\n \\\\\\\"resourceLocation\\\\\\\": \\\\\\\"/modelmanagement/v1.0/subscriptions/f6b17c2b-afa1-437e-9b38-3c41d81dbac2/resourceGroups/mrg-huggingface-20220609090948/providers/Microsoft.MachineLearningServices/workspaces/distilberttestkfealm5afkpq4/onlineEndpoints/distilberttestkfealm5afkep1\\\\\\\",\\\\r\\\\n \\\\\\\"operationLog\\\\\\\": \\\\\\\"\\\\\\\",\\\\r\\\\n \\\\\\\"parentRequestId\\\\\\\": \\\\\\\"635b702a-d27e-4408-b6b3-f589c57d66d6\\\\\\\"\\\\r\\\\n }\\\\r\\\\n }\\\\r\\\\n ]\\\\r\\\\n }\\\\r\\\\n ]\\\\r\\\\n }\\\\r\\\\n}\\\"}]}}. Please refer to https://aka.ms/DeploymentScriptsTroubleshoot for more deployment script information.\"\r\n      }\r\n    ]\r\n  }\r\n}"
            }
        ]
    }
}

What is interesting in this status message is the “Resource provider [N/A] isn’t registered with Subscription [N/A]”. However, I am not sure how to interpret it.

philschmid · June 9, 2022, 5:02pm

@Pbrezina could you go on the azure portal to subscription → select your subscription → navigate to “resource providers” (left side) → search for “machine” → check if “Microsoft.MachineLearningServices” are “registered”

Pbrezina · June 10, 2022, 6:03am

@philschmid I checked the resource providers and I do have Microsoft.MachineLearningServices and Microsoft.MachineLearning registered.

philschmid · June 15, 2022, 9:38am

@Pbrezina we are heavily working with the Microsoft team to make the experience better. It is strange to see that it still doesn’t work and the error is saying N/A provider.

Could you check if Microsoft.Authorization is also registered?

Thank you for your help and testing the preview

Pbrezina · June 16, 2022, 6:44am

@philschmid yes, Microsoft.Authorization is registered as well.

thomasvrancken · July 4, 2022, 2:24pm

I actually ran in the exact same issue. I have a similar setup (blank new project, paid subscription, copy pasted the options from the blogpost).

I could replicate this error log:

{
    "status": "Failed",
    "error": {
        "code": "ApplianceDeploymentFailed",
        "message": "The operation to create appliance failed. Please check operations of deployment 'hugtextclass' under resource group '/subscriptions/f6b17c2b-afa1-437e-9b38-3c41d81dbac2/resourceGroups/mrg-huggingface-20220609090948'. Error message: 'At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.'",
        "details": [
            {
                "code": "Conflict",
                "message": "{\r\n  \"status\": \"failed\",\r\n  \"error\": {\r\n    \"code\": \"ResourceDeploymentFailure\",\r\n    \"message\": \"The resource operation completed with terminal provisioning state 'failed'.\",\r\n    \"details\": [\r\n      {\r\n        \"code\": \"DeploymentScriptError\",\r\n        \"message\": \"The provided script failed with the following error:\\r\\nERROR: {\\\"status\\\":\\\"Failed\\\",\\\"error\\\":{\\\"code\\\":\\\"DeploymentFailed\\\",\\\"message\\\":\\\"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.\\\",\\\"details\\\":[{\\\"code\\\":\\\"Conflict\\\",\\\"message\\\":\\\"{\\\\r\\\\n \\\\\\\"status\\\\\\\": \\\\\\\"Failed\\\\\\\",\\\\r\\\\n \\\\\\\"error\\\\\\\": {\\\\r\\\\n \\\\\\\"code\\\\\\\": \\\\\\\"ResourceDeploymentFailure\\\\\\\",\\\\r\\\\n \\\\\\\"message\\\\\\\": \\\\\\\"The resource operation completed with terminal provisioning state 'Failed'.\\\\\\\",\\\\r\\\\n \\\\\\\"details\\\\\\\": [\\\\r\\\\n {\\\\r\\\\n \\\\\\\"message\\\\\\\": \\\\\\\"SubscriptionNotRegistered: Resource provider [N/A] isn't registered with Subscription [N/A]. Please see troubleshooting guide, available here: https://aka.ms/register-resource-provider\\\\\\\",\\\\r\\\\n \\\\\\\"details\\\\\\\": [],\\\\r\\\\n \\\\\\\"additionalInfo\\\\\\\": [\\\\r\\\\n {\\\\r\\\\n \\\\\\\"type\\\\\\\": \\\\\\\"MmsOperation\\\\\\\",\\\\r\\\\n \\\\\\\"info\\\\\\\": {\\\\r\\\\n \\\\\\\"id\\\\\\\": \\\\\\\"279fddd5-8685-4e69-b175-a4a7c027dc1b\\\\\\\",\\\\r\\\\n \\\\\\\"operationType\\\\\\\": \\\\\\\"OnlineEndpoint\\\\\\\",\\\\r\\\\n \\\\\\\"state\\\\\\\": \\\\\\\"Failed\\\\\\\",\\\\r\\\\n \\\\\\\"createdTime\\\\\\\": \\\\\\\"2022-06-09T07:16:09.8692253Z\\\\\\\",\\\\r\\\\n \\\\\\\"endTime\\\\\\\": \\\\\\\"2022-06-09T07:16:30.929291Z\\\\\\\",\\\\r\\\\n \\\\\\\"error\\\\\\\": {\\\\r\\\\n \\\\\\\"message\\\\\\\": \\\\\\\"SubscriptionNotRegistered: Resource provider [N/A] isn't registered with Subscription [N/A]. Please see troubleshooting guide, available here: https://aka.ms/register-resource-provider\\\\\\\"\\\\r\\\\n },\\\\r\\\\n \\\\\\\"resourceLocation\\\\\\\": \\\\\\\"/modelmanagement/v1.0/subscriptions/f6b17c2b-afa1-437e-9b38-3c41d81dbac2/resourceGroups/mrg-huggingface-20220609090948/providers/Microsoft.MachineLearningServices/workspaces/distilberttestkfealm5afkpq4/onlineEndpoints/distilberttestkfealm5afkep1\\\\\\\",\\\\r\\\\n \\\\\\\"operationLog\\\\\\\": \\\\\\\"\\\\\\\",\\\\r\\\\n \\\\\\\"parentRequestId\\\\\\\": \\\\\\\"635b702a-d27e-4408-b6b3-f589c57d66d6\\\\\\\"\\\\r\\\\n }\\\\r\\\\n }\\\\r\\\\n ]\\\\r\\\\n }\\\\r\\\\n ]\\\\r\\\\n }\\\\r\\\\n}\\\"}]}}. Please refer to https://aka.ms/DeploymentScriptsTroubleshoot for more deployment script information.\"\r\n      }\r\n    ]\r\n  }\r\n}"
            }
        ]
    }
}

With the surprising “Resource provider [N/A] isn’t registered with Subscription [N/A]”.

@Pbrezina Could you find a solution to it?

philschmid · July 5, 2022, 11:24am

@thomasvrancken when have you tried the deployment? there should be a fixed rolled out on friday

thomasvrancken · July 6, 2022, 6:09am

@philschmid Actually it was a couple of weeks ago. Great to know there is a fix being rolled out! I’ll try again after Friday then

Pbrezina · July 11, 2022, 7:06am

Hello,

Just FYI. I’ve just tried to deploy the Huggingface endpoint with finetuned distilbert model and the resource was successfully created!

So it seems the issue has been resolved.

philschmid · July 11, 2022, 7:38am

@Pbrezina that’s great to hear! Are you planning to use the endpoints if so for what task/model?

Pbrezina · July 11, 2022, 8:04am

@philschmid No, I will not. I’m working on text extraction from scanned semi-structured documents, like invoices, receipts and such. For this task, I intend to use LayoutLM model, but as you’ve pointed out earlier, the model is not finetuned and token classification currently doesn’t support non NLP models, so LayoutLM can’t be used with the Huggingface endpoint on Azure. But I’ll keep this option in mind and if I’ll work on purely NLP task in the future, I’ll definitely try out the Huggingface Endpoint

philschmid · July 12, 2022, 7:03am

I am happy to share that for GA of the service we will support custom pipelines.py for models like LayoutLM. For this you would basically define a pipeline.py file in the model repository on Models - Hugging Face quite similar to the transformers pipelines and can then use if for inference.