AzureBlobStorage AuthorizationPermissionMismatch error using User Managed Identities, Python SDK from AKS – Debian

I’m running a Python app in AKS (as a Job, but doesn’t matter), using the Azure Python SDK to access blob storage. I’m using a User Managed Identity for auth, using ManagedIdentityCredential with the client_id kwarg (see https://learn.microsoft.com/en-us/python/api/azure-identity/azure.identity.managedidentitycredential?view=azure-python). It is successfully able to query the IMDS endpoint and obtain a token, but I’m still hitting an error. Anybody has any idea about what setup I might be missing?

There are precious little docs about user managed identities overall, esp in relation to AKS and the blob store, and this error.

Successful IMDS token fetch:

2021-01-26 05:26:05,944 - azure.core.pipeline.policies.http_logging_policy - INFO - Request URL: 'http://REDACTED/metadata/identity/oauth2/token?api-version=REDACTED&resource=REDACTED&client_id=REDACTED'
2021-01-26 05:26:05,945 - azure.core.pipeline.policies.http_logging_policy - INFO - Request method: 'GET'
2021-01-26 05:26:05,945 - azure.core.pipeline.policies.http_logging_policy - INFO - Request headers:
2021-01-26 05:26:05,945 - azure.core.pipeline.policies.http_logging_policy - INFO -     'Metadata': 'REDACTED'
2021-01-26 05:26:05,945 - azure.core.pipeline.policies.http_logging_policy - INFO -     'User-Agent': 'azsdk-python-identity/1.5.0 Python/3.7.7 (Linux-4.15.0-1103-azure-x86_64-with-debian-9.12)'
2021-01-26 05:26:05,945 - azure.core.pipeline.policies.http_logging_policy - INFO - No body was attached to the request
2021-01-26 05:26:05,956 - azure.core.pipeline.policies.http_logging_policy - INFO - Response status: 200
2021-01-26 05:26:05,956 - azure.core.pipeline.policies.http_logging_policy - INFO - Response headers:
2021-01-26 05:26:05,956 - azure.core.pipeline.policies.http_logging_policy - INFO -     'Content-Type': 'application/json; charset=utf-8'
2021-01-26 05:26:05,956 - azure.core.pipeline.policies.http_logging_policy - INFO -     'Server': 'IMDS/150.870.65.486'
2021-01-26 05:26:05,956 - azure.core.pipeline.policies.http_logging_policy - INFO -     'Date': 'Tue, 26 Jan 2021 05:26:05 GMT'
2021-01-26 05:26:05,956 - azure.core.pipeline.policies.http_logging_policy - INFO -     'Content-Length': '1760'
2021-01-26 05:26:05,957 - azure.identity._internal.decorators - INFO - ManagedIdentityCredential.get_token succeeded
2021-01-26 05:26:05,957 - azure.identity._credentials.chained - INFO - ChainedTokenCredential acquired a token from ManagedIdentityCredential

Subsequent API call to blob.core.windows.net/…. errors:

  File "/usr/local/lib/python3.7/site-packages/azure/storage/blob/_blob_client.py", line 685, in upload_blob
    return upload_block_blob(**options)
  File "/usr/local/lib/python3.7/site-packages/azure/storage/blob/_upload_helpers.py", line 157, in upload_block_blob
    process_storage_error(error)
  File "/usr/local/lib/python3.7/site-packages/azure/storage/blob/_shared/response_handlers.py", line 150, in process_storage_error
    error.raise_with_traceback()
  File "/usr/local/lib/python3.7/site-packages/azure/core/exceptions.py", line 218, in raise_with_traceback
    raise super(AzureError, self).with_traceback(self.exc_traceback)
  File "/usr/local/lib/python3.7/site-packages/azure/storage/blob/_upload_helpers.py", line 105, in upload_block_blob
    **kwargs)
  File "/usr/local/lib/python3.7/site-packages/azure/storage/blob/_generated/operations/_block_blob_operations.py", line 233, in upload
    raise HttpResponseError(response=response, model=error)
azure.core.exceptions.HttpResponseError: This request is not authorized to perform this operation using this permission.
RequestId:defcc13f-101e-006c-6aa3-f321cb000000
Time:2021-01-26T05:26:06.0112926Z
ErrorCode:AuthorizationPermissionMismatch
Error:None

The blob storage resource in question has a role assignment to the user-managed identity, as a "contributor" to "This resource".

Code:

    managed_identity = ManagedIdentityCredential(client_id=mi_client_id)
    azure_cli = AzureCliCredential()
    credential_chain = ChainedTokenCredential(managed_identity, azure_cli)
    return BlobServiceClient(url_prefix, credential=credential_chain)

Versions:
azure-identity 1.5, Python 3.7.7

I’m not sure our AKS Cluster has granted the user managed identity a role, and I’m not sure if that matters, or what else needs to be set up.

thanks