skip to Main Content

I am trying to learn terraform, by creating a kubernetes cluster and a container registry in azure and giving the kubernetes cluster pull access to the container registry.

I have manually created a service principal with a custom role and authenticated it like they do in the terraform azure totorial. The custom role has all contributor permissions along with the following

Microsoft.Authorization/roleAssignments/read
Microsoft.Authorization/roleAssignments/write
Microsoft.Authorization/roleAssignments/delete
Microsoft.ContainerService/managedClusters/read
Microsoft.ContainerService/managedClusters/write
Microsoft.ContainerService/managedClusters/delete

And I have correctly set the env vars. When I then try to terraform apply the following file:

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 3.104.2"
    }
  }

  required_version = ">= 1.1.0"
}

provider "azurerm" {
  features {

  }
}

resource "azurerm_resource_group" "rg" {
  name     = "myRG"
  location = "North Europe"
}

resource "azurerm_container_registry" "acr" {
  name                = "mycr"
  resource_group_name = azurerm_resource_group.rg.name
  location            = azurerm_resource_group.rg.location
  sku                 = "Basic"
}

resource "azurerm_kubernetes_cluster" "aks" {
  name                = "myAKS"
  location            = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  dns_prefix          = "myAKS"

  default_node_pool {
    name       = "default"
    node_count = 1
    vm_size    = "Standard_D2_v2"
  }

  identity {
    type = "SystemAssigned"
  }
}

# Attach the container registry to the kubernetes cluster
resource "azurerm_role_assignment" "aksPullFromAcr" {
  principal_id         = azurerm_kubernetes_cluster.aks.kubelet_identity[0].object_id
  role_definition_name = "AcrPull"
  scope                = azurerm_container_registry.acr.id
}

I get the following error. The weird thing being that the subscription id is pointing to the wrong subscription

Error: retrieving Kubernetes Cluster (Subscription: "<wrong_subscription_id>"
│ Resource Group Name: "myRG"
│ Kubernetes Cluster Name: "myAKS"): unexpected status 403 (403 Forbidden) with error: AuthorizationFailed: The client '<client_id>' with object id '<client_id>' does not have authorization to perform action 'Microsoft.ContainerService/managedClusters/read' over scope '/subscriptions/<wrong_subscription_id>/resourceGroups/myRG/providers/Microsoft.ContainerService/managedClusters/myAKS' or the scope is invalid. If access was recently granted, please refresh your credentials.

I have tried to make a new service principal, but that didn’t help.

What am I doing wrong here?

2

Answers


  1. Chosen as BEST ANSWER

    I ended up deleting the terraform.tfstate file which fixed the issue.

    The az account show already showed the right subscription, and the other solutions didn't help either.


  2. I get the following error. The weird thing being that the subscription id is pointing to the wrong subscription.

    If you are authenticating Terraform with the service principal by setting the environment variables and Terraform is going to another subscription instead of the correct subscription specified in the environment variables, you need to check the following steps.

    1. Check if terminal is already logged in with a different account. You may check the login status by using the command: az account show. If logged in, kindly log out using az logout and try executing the terraform apply command.

    2. Make sure to check the CLIENT_ID, CLIENT_SECRET, SUBSCRIPTION_ID, and TENANT_ID that are configured in the environment variables for authentication are the same.

    3. To set to correct subscription, run the below cmd.

    az account set --subscription  "your current Sub_ID"
    
    1. If you are still facing an error, you may also try logging in with the service principal using the Azure CLI.
    az login --service-principal -u CLIENT_ID -p CLIENT_SECRET --tenant TENANT_ID
    
    provider "azurerm" {
      features {}
      subscription_id   = "158b8345-c359-4d98-95c5-f21815dd048f"
      tenant_id         = "226cf998-ddcc-4005-acfb-9bbfa7d40283"
      client_id         = "bf7e17bd-d24f-4273-8f79-10f90c4811d2"
      client_secret     = "ibf8Q~1sBiX36uuEKpQvt11JJkEECNpmoVZ6BaOD"
    }
    
    resource "azurerm_resource_group" "rg" {
      name     = "myRG"
      location = "North Europe"
    }
    
    resource "azurerm_container_registry" "acr" {
      name                = "sampleacr"
      resource_group_name = azurerm_resource_group.rg.name
      location            = azurerm_resource_group.rg.location
      sku                 = "Basic"
    }
    
    resource "azurerm_kubernetes_cluster" "aks" {
      name                = "myAKS"
      location            = azurerm_resource_group.rg.location
      resource_group_name = azurerm_resource_group.rg.name
      dns_prefix          = "myAKS"
    
      default_node_pool {
        name       = "default"
        node_count = 1
        vm_size    = "Standard_D2_v2"
      }
    
      identity {
        type = "SystemAssigned"
      }
    }
    resource "azurerm_role_assignment" "aksPullFromAcr" {
      principal_id         = azurerm_kubernetes_cluster.aks.kubelet_identity[0].object_id
      role_definition_name = "AcrPull"
      scope                = azurerm_container_registry.acr.id
    }
    

    Terraform init

    As there is no account logged in to the terminal, Terraform is authenticating with the service principal specified in the provider.

    enter image description here

    Terraform apply

    enter image description here

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search