Question 247205 in Azure Question posted in
The official documentation to get you started can be found here.

Azure – databricks deployment bicep suddenly failing

The following bicep deployment has suddenly started failing and I do not understand why

var managedResourceGroupName = 'rg-metadata-${dataBricksName}'

resource databrickworkspace 'Microsoft.Databricks/workspaces@2018-04-01' = {
  name: dataBrickName
  location: location
  sku: {
    name: dataBrickPricingTier
  }
  properties: {
    managedResourceGroupId: managedResourceGroup.id
    parameters: {
      enableNoPublicIp: {
        value: disablePublicIp
      }
    }
  }
}

resource managedResourceGroup 'Microsoft.Resources/resourceGroups@2021-04-01' existing = {
  scope: subscription()
  name: managedResourceGroupName
}

This pretty much follows the same doc : here.

The error message I get is

The filter 'principalId eq ''' is not supported. Supported filters are either 'atScope()' or 'principalId eq '{value}' or assignedTo('{value}')

This error message is quite cyrptic and does not tell me much. Also, this has been succeeding so far. The problem has started now.

Looking at the full doc here , I can see it has things like

enter image description here

enter image description here

It says required, but the definition isn’t clear as to what do I have to put there. I have never had to use the principalId so far, during the creation of a databricks workspace.

Another fun fact is: the deletion of the resource (databricks workspace which is anyways in a failed state), does not go through and fails too or gets stuck.

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    I raised a MS Support case for this, as I ended up spending a lot of hours on it. Also, it wasn't something wrong that I was doing- especially because it worked in the past and also, that the same bicep worked flawlessly the next morning.

    MS supporthave acknowledged it was a service failure (details below). So, if some of you encounter these cryptic errors and have no idea, what could be going wrong, raise a support case.

    **Summary of Impact**: 
A few users using Azure Databricks may have experienced failures during Workspace creation with 
Premium Pricing Tier.
 
**Preliminary Root Cause**: 
We determined that a new feature deployed in a dependent backend service, intended to aid 
in the mapping of new workspaces, encountered rate limit issues during the resource provisioning 
process, causing Workspace creation failures as mentioned above.
Mitigation: We performed a rollback of the new feature deployment to a previously known good 
version to mitigate the issue and restore functionality. Additionally, service health has been 
closely monitored to ensure no failures are detected before the issue was declared as fully 
mitigated.Full-service functionality has been confirmed.

  2. 0

    By referring to the same MSDoc provided by you, I tried deploying databricks with workspace in my environment and was able to deploy it successfully.

    param disablePublicIp bool = false
param workspaceName string = 'newwsj'
param Tier string = 'premium'
param location string = resourceGroup().location

var managedResourceGroupName = 'databricks-rg-${workspaceName}-${uniqueString(workspaceName, resourceGroup().id)}'

resource ws 'Microsoft.Databricks/workspaces@2018-04-01' = {
  name: workspaceName
  location: location
  sku: {
    name: Tier
  }
  properties: {
    managedResourceGroupId: managedResourceGroup.id
    parameters: {
      enableNoPublicIp: {
        value: disablePublicIp
      }
    }
  }
}

resource managedResourceGroup 'Microsoft.Resources/resourceGroups@2021-04-01' existing = {
  scope: subscription()
  name: managedResourceGroupName
}

    Deployment succeeded:

    enter image description here

    enter image description here

    When I open the JSON view of the deployed resource, it displays the provisioningstate as successful with the specified principalId. In addition to the databricks account, the above bicep code generates a managed resource group, which includes an identity and the service principal details. Only at this page, the service principal details can be viewed.

    Json view:

    enter image description here

    Path to view the service principal details:

    Managed resource group created for databricks-> Managed identity -> Properties

    enter image description here

    And

    Regarding the resource’s deletion, it’s probably that it’s stuck in a failed state, preventing it from being destroyed. You can see if removing the resource group that contains the resource solves the issue.

    Note: Verify the managed resource group name from the document and provide in your code correctly. Sometimes the variable names also create a conflict for deployment.

    For more information: Refer Databricks service principal document

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search