We are developing a GitHub Action to deploy topics, ACLs, and connectors using the Terraform Confluent provider. Authentication is handled through a Service Principal (SPN), and the secrets, such as the SPN’s secret or Kafka API keys, are retrieved from an Azure KeyVault.
Our provider configuration looks like this:
terraform {
required_providers {
confluent = {
source = "confluentinc/confluent"
version = "1.77.0"
}
azurerm = {
source = "hashicorp/azurerm"
version = "~> 3.95"
}
}
backend "azurerm" {
resource_group_name = "my_resource_group"
storage_account_name = "mystorageacc"
container_name = "terraform"
key = "connectors/ccpre.tfstate"
client_id = data.azurerm_key_vault_secret.client_id.value
client_secret = data.azurerm_key_vault_secret.client_secret.value
tenant_id = data.azurerm_key_vault_secret.tenant_id.value
subscription_id = data.azurerm_key_vault_secret.subscription_id.value
}
}
provider "azurerm" {
features {}
}
provider "confluent"{
cloud_api_key = data.azurerm_key_vault_secret.confluent_cloud_api_key.value
cloud_api_secret = data.azurerm_key_vault_secret.confluent_cloud_api_secret.value
}
We’ve been researching options for securing the tfstate files and it seems that other cloud providers like S3 (AWS) and GCS (Google Cloud Storage) support encryption options that help protect sensitive data in tfstate files https://developer.hashicorp.com/terraform/language/state/sensitive-data.
However, we haven’t found a clear way to apply similar encryption in Azure Blob Storage without potentially disrupting the standard Terraform plan and apply workflows. We’re looking for a way to enable encryption on Azure Blob storage that is compatible with Terraform’s requirements or for alternative approaches that ensure tfstate remains secure.
2
Answers
You can encrypt the
Terraform state
file in Azure in a more secure way.Azure-managed keys
orcustomer-managed keys
.This way, when you use sensitive information such as client secrets, client ID, subscription ID, and tenant ID, all will be securely stored in the Azure storage account with encryption.
Assign a role such as Storage Blob Data Contributor to the service principal or user to access the storage account, ensuring that only authorized individuals can access the state file.
Reference: Securing Terraform State in Azure by
Chris_Ayers
Azure Storage encryption for data at rest
Server-side encryption of Azure Disk Storage
Terraform just released v1.10. There are new resources called ephemeral resources.
Ephemeral resources’s values never gets stored into state file.
It is meant to solve the problem you are facing of storing sensitive value to state file.
https://developer.hashicorp.com/terraform/language/v1.10.x/resources/ephemeral