Terraform tasks to deploy on azure cloud

Assuming your Azure Resource manager service connection is created with the underlying service principal (app registration not a managed identity), as far as I have tested to use the TerraformTaskV4@4 task to initialize terraform backend state in Azure storage account, we only need to make sure the underlying service principal (app registration) of the Azure Resource Manager service connection is assigned with the contributor role to the scope of the subscription.

Besides, please remove the options to use backendAzureRmUseEnvironmentVariablesForAuthenticationand backendAzureRmUseEntraIdForAuthentication, since the task will use the service principal information stored in the ARM service connection to authenticate access to Azure subscription and storage account.

- task: TerraformTaskV4@4
  displayName: terraform init
  inputs:
    provider: 'azurerm'
    command: 'init'
    workingDirectory: '$(System.DefaultWorkingDirectory)/RGEmptyBackend'
    # backendAzureRmUseEnvironmentVariablesForAuthentication: true
    # backendAzureRmUseEntraIdForAuthentication: true
    backendServiceArm: 'ARMSvcCnnWIFTerraform'
    backendAzureRmResourceGroupName: '$(ARM_RESOURCE_GROUP_NAME)'
    backendAzureRmStorageAccountName: '$(ARM_STORAGE_ACCOUNT_NAME)'
    backendAzureRmContainerName: '$(ARM_CONTAINER_NAME)'
    backendAzureRmKey: 'rg/rgemptybackend.tfstate'

You may follow the marks in the screenshots below to double check if the underlying service principal (app registration) is granted with sufficient permissions, redirecting from the ARM service connection in your Project Settings (ARMSvcCnnWIFTerraform in my case) that is referenced by this task. If your ARM service connection was created with service principal client id and secret rather than wiht workload identity federation authentication, you may also try to Edit and Verify the service connection.

If you happen to know the ARM_CLIENT_ID and ARM_CLIENT_SECRET etc. information of the service principle, you may consider using a simple script for terraform init as a possible workaround or as a method to double check principal permissions in your local environment or via pipelines. Here are the samples for your reference.

# Configure Terraform

variable "RGNAME" {
  type = string
}

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "3.42.0"
    }
    azuread = {
      source  = "hashicorp/azuread"
      version = "~> 2.0"
    }
  }
  backend "azurerm" {}
}

provider "azurerm" {
  features {}
}

resource "azurerm_resource_group" "rg" {
  name     = var.RGNAME
  location = "Southeast Asia"
}

output "test" {
    value = azurerm_resource_group.rg.id
}

variables:
- name: ARM_CLIENT_ID
  value: 2d06022b-xxxxxx
- ARM_CLIENT_SECRET
  value: xxxxxx
- ARM_SUBSCRIPTION_ID
  value: xxxxxx
- ARM_TENANT_ID
  value: xxxxxx

- name: ARM_RESOURCE_GROUP_NAME
  value: rg-azstorageaccount
- name: ARM_STORAGE_ACCOUNT_NAME
  value: azstorageaccountxxxxxx
- name: ARM_CONTAINER_NAME
  value: containerxxxxxx

pool:
  vmImage: ubuntu-latest

steps:
- task: TerraformInstaller@1
  inputs:
    terraformVersion: 'latest'
- script: |
    terraform init -backend-config="resource_group_name=$(ARM_RESOURCE_GROUP_NAME)" -backend-config="storage_account_name=$(ARM_STORAGE_ACCOUNT_NAME)" -backend-config="container_name=$(ARM_CONTAINER_NAME)" -backend-config="key=rg/rg-sp-emptybackend.tfstate"
    terraform apply -auto-approve -input=false -var "RGNAME=${{ parameters.TF_VAR_RGNAME }}-$(Build.BuildId)"
  workingDirectory: '$(System.DefaultWorkingDirectory)/RGEmptyBackend'
  env:
    ARM_CLIENT_ID: $(ARM_CLIENT_ID)
    ARM_CLIENT_SECRET: $(ARM_CLIENT_SECRET)
    ARM_SUBSCRIPTION_ID: $(ARM_SUBSCRIPTION_ID)
    ARM_TENANT_ID: $(ARM_TENANT_ID)