skip to Main Content

I see weird behavior in my terraform plan and apply.
I’m creating a aws_security_group resource with two ingress block settings and one dynamic ingress block, for the ingress block I’m passing as variable the additional sg rules.

resource "aws_security_group" "db" {
  name_prefix            = "${var.name}-db-"
  description            = "Allow db access"
  vpc_id                 = var.vpc_id
  revoke_rules_on_delete = false

  ingress {
    from_port       = 3306
    to_port         = 3306
    description     = "X"
    protocol        = "tcp"
    security_groups = [var.x-sg]
  }

  ingress {
    from_port   = 3306
    to_port     = 3306
    protocol    = "tcp"
    description = "Y"
    cidr_blocks = [var.cidr_block]
  }

  dynamic "ingress" {
    for_each = var.db_sg_ingress_rules
    content {
      from_port   = ingress.value["port"]
      to_port     = ingress.value["port"]
      protocol    = ingress.value["protocol"]
      description = ingress.value["description"]
      cidr_blocks = ingress.value["cidr_blocks"]
    }
  }
}

All the var passed correctly, setting var:

db_sg_ingress_rules = [{
        port        = 3306
        protocol    = "tcp"
        description = "A"
        cidr_blocks = ["10.10.0.0/16"]
    },
    {
        port        = 3306
        protocol    = "tcp"
        description = "B"
        cidr_blocks = ["180.20.0.0/16"]
    },
    {
        port        = 3306
        protocol    = "tcp"
        description = "C"
        cidr_blocks = ["12.11.10.0/24"]
    }]

configure this way:

variable "db_sg_ingress_rules" {
  type = list(object({
    port        = number
    protocol    = string
    description = string
    cidr_blocks = list(string)
  }))
}

Once I run the terraform apply and approve it, I see each time the same suggestion for changes, terraform is trying to remove empty rule cidr_block definition:

  # module.security_group.aws_security_group.db will be updated in-place
  ~ resource "aws_security_group" "db" {
        id                     = "sg-xxxxxxxxxxxxxx"
      ~ ingress                = [
          - {
              - cidr_blocks      = []
              - description      = "B"
              - from_port        = 3306
              - ipv6_cidr_blocks = []
              - prefix_list_ids  = []
              - protocol         = "tcp"
              - security_groups  = []
              - self             = false
              - to_port          = 3306
            },
            # (5 unchanged elements hidden)
        ]
        name                   = "db-xxxxxxxxxxxxxxx"
        tags                   = {}
        # (8 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

Another thing that I see is that if I ran it multiple times without changing any code the plan returns the same changes but for a different rule, for example not description = "B", but with description = "X".
The terraform state show for this resource not showing any empty definition.
I don’t want to use the aws_security_group_rule resources as I’m looking to manage the sg resource to see any drifts in the sg.

How can I solve this issue, I would like to see the plan empty without any changes.

I have tried to sort the list so it will be passed in the same order each time with:

sorted_values = sort(var.db_sg_ingress_rules[*].description)
sorted_list = tolist(flatten(
    [for value in local.sorted_values:
        [ for elem in var.db_sg_ingress_rules: 
              elem if value == elem.description
        ]     
    ]))

and then pass the sorted_list as the var to for_each.

UPDATE:
I have tried to combine all rules into one list and pass it into a singly dynamic block as well as tried to separate all of them into 5 blocks of ingress rules(under the security group resource), but I still see the same pattern with the empty rule that is being removed in terraform plan/apply.

2

Answers


  1. I am pretty sure the mix between the ingress blocks and the dynamic is causing your issue.

    Using local variable is a good way to validate it. Try to use only one dynamic block by creating a local variable containing all your ingress rules :

        locals {
          l_db_sg_ingress_rules = concat(var.db_sg_ingress_rules, 
          [
            {
              port        = 3306
              protocol    = "tcp"
              description = "X"
              cidr_blocks = [var.x-sg]
            },
            {
              port        = 3306
              protocol    = "tcp"
              description = "Y"
              cidr_blocks = [var.cidr_block]
            }
          ]
          ) 
        }
        
        resource "aws_security_group" "db" {
          name_prefix            = "${var.name}-db-"
          description            = "Allow db access"
          vpc_id                 = var.vpc_id
          revoke_rules_on_delete = false
    
          dynamic "ingress" {
            for_each = local.l_db_sg_ingress_rules
            content {
              from_port   = ingress.value["port"]
              to_port     = ingress.value["port"]
              protocol    = ingress.value["protocol"]
              description = ingress.value["description"]
              cidr_blocks = ingress.value["cidr_blocks"]
            }
          }
        }
    

    Let me know it has resolved your issue.

    Login or Signup to reply.
  2. From what I see in the provider documentation https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group

    Terraform currently provides a Security Group resource with ingress and egress rules defined in-line and a Security Group Rule resource which manages one or more ingress or egress rules. Both of these resource were added before AWS assigned a security group rule unique ID, and they do not work well in all scenarios using the description and tags attributes, which rely on the unique ID.

    It seems to me this is the source of the problem (perhaps in conjunction with utilizing of the dynamic block mentioned earlier) you are facing. You mentioned you do not want to use aws_security_group_rule resource, however:

    The aws_vpc_security_group_egress_rule and aws_vpc_security_group_ingress_rule resources have been added to address these limitations and should be used for all new security group rules

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search