Terraform: Dynamically Create AWS EFS Mount Targets – Amazon Web Sevices

OK. I admit it. I’m not able to figure this out. I really really really want to like Terraform but the more I play around with it the more frustrated I’m getting and eyeing AWS CDK a little more seriously each time.

The Situation

1. I am creating one public and one private subnet per each availability zone in a region. I don’t necessarily know which region I’m going to deploy my resources to so I use a VPC data block for this:

    data "aws_availability_zones" "available" {
        state = "available"
    }

2. This is done in a module named, ‘vpc’. I then go off and create an EFS file system in another module named, ‘efs’. Inside of that ‘efs’ module, I am trying to create a set of EFS mount targets – one per each of my private subnets:

resource "aws_efs_mount_target" "efs_mount_targets_private" {
    for_each = toset(var.private_subnets)

    file_system_id = aws_efs_file_system.pgsql_databases.id
    security_groups = [ "${var.allow-nfs-ingress-my-vpc}" ]
    subnet_id = each.value
}

Some of you can guess what’s coming, can’t you?

3. A terraform plan fails:

Error: Invalid for_each argument
│ 
│   on modules/efs/main.tf line 55, in resource "aws_efs_mount_target" "efs_mount_targets_private":
│   XX:     for_each = toset(var.private_subnets)
│     ├────────────────
│     │ var.private_subnets is a list of string, known only after apply

The Code

Partial main.tf (root):

module "vpc" {
  count                              = terraform.workspace == "backend" ? 0 : 1
  source                             = "./modules/vpc"

  vpc_flow_logs_to_cloud_watch_group = module.cw[0].my-vpc-flow-logs-group
  vpc_flow_logs_to_cloud_watch_role  = module.iam[0].vpc_flow_logs_to_cloud_watch_role
  vpn_vgw_id = module.vpn[0].vpn-vgw-id
  nat_gw_eip = module.ec2[0].nat_gw_eip
}

module "efs" {
  count = terraform.workspace == "backend" ? 0 : 1
  source = "./modules/efs"

  vpc_id = module.vpc[0].my_vpc_id
  allow-nfs-ingress-my-vpc = module.sg-ingress[0].allow-nfs-ingress-my-vpc-id
  private_subnets = module.vpc[0].my_vpc_private_subnets
}

Partial main.tf (vpc module):

# Create private subnets out of odd octets
resource "aws_subnet" "my_vpc_private_subnets" {
  for_each = toset(data.aws_availability_zones.available.names)

  vpc_id = aws_vpc.my_vpc.id
  availability_zone = each.value
  cidr_block = "172.25.${2*(index(data.aws_availability_zones.available.names, each.value))+1}.0/24"

  tags = {
    auto-delete = "no"
    Name = "${each.value} subnet ${2*(index(data.aws_availability_zones.available.names, each.value))+1}"
    tier = "private"
  }
}

data "aws_subnets" "private_subnets" {
  filter {
    name = "vpc-id"
    values = [aws_vpc.my_vpc.id]
  }

  tags = {
    tier = "private"
  }

  # Without the below, this data block might evaluate to empty
  depends_on = [
    aws_subnet.my_vpc_private_subnets
  ]
}

Partial outputs.tf (vpc module):

output "my_vpc_private_subnets" {
  value = data.aws_subnets.private_subnets.ids
}

Partial main.tf (efs module):

resource "aws_efs_mount_target" "efs_mount_targets_private" {
    for_each = toset(var.private_subnets)

    file_system_id = aws_efs_file_system.pgsql_databases.id
    security_groups = [ "${var.allow-nfs-ingress-my-vpc}" ]
    subnet_id = each.value
}

The Question
Surely(?) there’s a way to get around this issue WITHOUT having to do two-stage apply? I have tried putting an aws_subnets data block inside of the ‘ifs’ module and it doesn’t make a bit of difference. Still get the same error.

If what I was trying to do wasn’t possible, then how was I able to build the subnets I need off of the aws_availability_zones data block to begin with?

Yes, there are several other types of questions like this that I’ve read over but none of the answers I’ve found so far are really clicking for me so I apologize in advance for this indulgence.