skip to Main Content

We are developing a GitHub Action to deploy topics, ACLs, and connectors using the Terraform Confluent provider. Authentication is handled through a Service Principal (SPN), and the secrets, such as the SPN’s secret or Kafka API keys, are retrieved from an Azure KeyVault.

Our provider configuration looks like this:

terraform {
    required_providers {
        confluent = {
            source = "confluentinc/confluent"
            version = "1.77.0"
        }
    azurerm = {
        source = "hashicorp/azurerm"
        version = "~> 3.95"
    }
}

backend "azurerm" {
    resource_group_name = "my_resource_group"
    storage_account_name = "mystorageacc"
    container_name = "terraform"
    key = "connectors/ccpre.tfstate"

    client_id = data.azurerm_key_vault_secret.client_id.value
    client_secret = data.azurerm_key_vault_secret.client_secret.value
    tenant_id = data.azurerm_key_vault_secret.tenant_id.value
    subscription_id = data.azurerm_key_vault_secret.subscription_id.value
 }
}

provider "azurerm" {
  features {}
}

provider "confluent"{
    cloud_api_key = data.azurerm_key_vault_secret.confluent_cloud_api_key.value
    cloud_api_secret = data.azurerm_key_vault_secret.confluent_cloud_api_secret.value
}

We’ve been researching options for securing the tfstate files and it seems that other cloud providers like S3 (AWS) and GCS (Google Cloud Storage) support encryption options that help protect sensitive data in tfstate files https://developer.hashicorp.com/terraform/language/state/sensitive-data.

However, we haven’t found a clear way to apply similar encryption in Azure Blob Storage without potentially disrupting the standard Terraform plan and apply workflows. We’re looking for a way to enable encryption on Azure Blob storage that is compatible with Terraform’s requirements or for alternative approaches that ensure tfstate remains secure.

2

Answers


  1. Protecting sensitive data in the Terraform state file in an Azure environment

    You can encrypt the Terraform state file in Azure in a more secure way.

    1. Create a container for the Terraform state file within an Azure storage account, and enable Azure Storage Service Encryption to secure your Terraform state files at rest using either Azure-managed keys or customer-managed keys.

    enter image description here

    This way, when you use sensitive information such as client secrets, client ID, subscription ID, and tenant ID, all will be securely stored in the Azure storage account with encryption.

    1. Enable RBAC for authentication in the storage account.

    enter image description here

    Assign a role such as Storage Blob Data Contributor to the service principal or user to access the storage account, ensuring that only authorized individuals can access the state file.

    1. You can allow the required device IPs in the firewall section, ensuring that only authorized devices have access.

    enter image description here

    Reference: Securing Terraform State in Azure by Chris_Ayers

    Azure Storage encryption for data at rest

    Server-side encryption of Azure Disk Storage

    Login or Signup to reply.
  2. We’ve been researching options for securing the tfstate files

    Terraform just released v1.10. There are new resources called ephemeral resources.

    Ephemeral resources’s values never gets stored into state file.

    It is meant to solve the problem you are facing of storing sensitive value to state file.

    https://developer.hashicorp.com/terraform/language/v1.10.x/resources/ephemeral

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search