Amazon Web Services Question posted in
The official Amazon Web Services documentation can be found here.

Amazon web services – How do I properly set up a cross-region transit gateway route with terraform?

I’m trying to set up a transit gateway (TGW) that spans two regions via terraform. I set up the two TGWs on either region, create a TGW attachment in either region, a route table with routes pointing to the opposite region with a defined subnet, but at creation time, there is an inherent default route table that points to the local subnet that messes with the routing. Here is a snippet of my terraform:

   resource "aws_ec2_transit_gateway" "main" {
    description = "main transit gateway"
    provider = aws

    tags = {
      Name = "main draas transit gateway"
    }
  }

  resource "aws_ec2_transit_gateway_vpc_attachment" "main" {
    subnet_ids = [aws_subnet.private.id]
    transit_gateway_id = aws_ec2_transit_gateway.main.id
    vpc_id = aws_vpc.main.id
    provider = aws

    tags = {
      Name = "main draas tgw attachment"
    }
  }

  resource "aws_ec2_transit_gateway_route_table" "main" {
    transit_gateway_id = aws_ec2_transit_gateway.main.id
    provider = aws

    tags = {
      Name = "main draas tgw route table"
    }
  }

  resource "aws_ec2_transit_gateway_route" "main" {
    destination_cidr_block = var.staging_private_subnet_cidr
    transit_gateway_route_table_id = aws_ec2_transit_gateway_route_table.main.id
    transit_gateway_attachment_id = aws_ec2_transit_gateway_vpc_attachment.main.id
    provider = aws
  }

  resource "aws_ec2_transit_gateway_peering_attachment" "main" {
    provider = aws
    peer_transit_gateway_id = aws_ec2_transit_gateway.dr.id
    transit_gateway_id = aws_ec2_transit_gateway.main.id
    peer_account_id = var.account_id
    peer_region = var.dr_region

    tags = {
      Name = "main draas tgw peering"
    }
  }

resource "aws_ec2_transit_gateway" "dr" {
  provider = aws.dr
  description = "dr transit gateway"

  tags = {
    Name = "dr transit draas gateway"
  }
}

resource "aws_ec2_transit_gateway_vpc_attachment" "dr" {
  provider = aws.dr
  subnet_ids = [aws_subnet.staging_private.id]
  transit_gateway_id = aws_ec2_transit_gateway.dr.id
  vpc_id = aws_vpc.staging.id

  tags = {
    Name = "dr tgw draas attachment"
  }
}

resource "aws_ec2_transit_gateway_route_table" "dr" {
  provider = aws.dr
  transit_gateway_id = aws_ec2_transit_gateway.dr.id

  tags = {
    Name = "dr tgw draas route table"
  }
}

resource "aws_ec2_transit_gateway_route" "dr" {
  provider = aws.dr
  destination_cidr_block = var.main_private_subnet_cidr
  transit_gateway_route_table_id = aws_ec2_transit_gateway_route_table.dr.id
  transit_gateway_attachment_id = aws_ec2_transit_gateway_vpc_attachment.dr.id
}

When I do a terraform apply, I get two TGW route tables in each region, the default, which points back to the CIDR that is associated with the TGW in the same region, and the one I create here in my terraform code, which should be routing traffic to the opposing region.

When I do a reachability analysis, an ec2 instance in us-east-1 cannot ping one in us-west-2 and the TGW route table is the problem. It seems as though the default TGW route table is being used while the one I create in terraform is ignored. I realize the way I accomplish peering in my .tf code isn’t correct, but even after I accept peering in the console, the routing is incorrect.

I know there are some limitations around terraform, but does anyone have any clever ways around this besides going into the AWS console and manually changing things?

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    It looks like if you turn the gateway route tables to point to data objects, it works. Confirmed with VPC Reachability Analyzer. I'll post updated code soon.

    EDIT: Updated with working Terraform:

      resource "aws_ec2_transit_gateway" "main" {
    description = "main transit gateway"
    provider = aws

    tags = {
      Name = "main draas transit gateway"
    }
  }

  resource "aws_ec2_transit_gateway_vpc_attachment" "main" {
    subnet_ids = [aws_subnet.private.id]
    transit_gateway_id = aws_ec2_transit_gateway.main.id
    vpc_id = aws_vpc.main.id
    provider = aws

    tags = {
      Name = "main draas tgw attachment"
    }
  }

  data "aws_ec2_transit_gateway_route_table" "main" {
    provider = aws

    filter {
      name   = "default-association-route-table"
      values = ["true"]
    }

    filter {
      name   = "transit-gateway-id"
      values = [aws_ec2_transit_gateway.main.id]
    }

    tags = {
      Name = "main tgw draas route table"
    }
  }

  resource "aws_ec2_transit_gateway_route" "main" {
    destination_cidr_block = var.staging_private_subnet_cidr
    transit_gateway_route_table_id = data.aws_ec2_transit_gateway_route_table.main.id
    transit_gateway_attachment_id = aws_ec2_transit_gateway_vpc_attachment.main.id
    provider = aws
  }

  resource "aws_ec2_transit_gateway_peering_attachment" "main" {
    provider = aws
    peer_transit_gateway_id = aws_ec2_transit_gateway.dr.id
    transit_gateway_id = aws_ec2_transit_gateway.main.id
    peer_account_id = var.account_id
    peer_region = var.dr_region

    tags = {
      Name = "main draas tgw peering"
    }
  }

## DR

resource "aws_ec2_transit_gateway" "dr" {
  provider = aws.dr
  description = "dr transit gateway"

  tags = {
    Name = "dr transit draas gateway"
  }
}

resource "aws_ec2_transit_gateway_vpc_attachment" "dr" {
  provider = aws.dr
  subnet_ids = [aws_subnet.staging_private.id]
  transit_gateway_id = aws_ec2_transit_gateway.dr.id
  vpc_id = aws_vpc.staging.id

  tags = {
    Name = "dr tgw draas attachment"
  }
}

data "aws_ec2_transit_gateway_route_table" "dr" {
  provider = aws.dr

  filter {
    name   = "default-association-route-table"
    values = ["true"]
  }

  filter {
    name   = "transit-gateway-id"
    values = [aws_ec2_transit_gateway.dr.id]
  }

  tags = {
    Name = "dr tgw draas route table"
  }
}

resource "aws_ec2_transit_gateway_route" "dr" {
  provider = aws.dr
  destination_cidr_block = var.main_private_subnet_cidr
  transit_gateway_route_table_id = data.aws_ec2_transit_gateway_route_table.dr.id
  transit_gateway_attachment_id = aws_ec2_transit_gateway_vpc_attachment.dr.id
}

data "aws_ec2_transit_gateway_peering_attachment" "dr" {
  provider = aws.dr
  depends_on = [ aws_ec2_transit_gateway_peering_attachment.main ]

  filter {
    name = "state"
    values = [ "pendingAcceptance" ]
  }

  # Only the second accepter/peer transit gateway is called from the peering attachment.
  filter {
    name = "transit-gateway-id"
    values = [ aws_ec2_transit_gateway_peering_attachment.main.peer_transit_gateway_id ]
  }
}

resource "aws_ec2_transit_gateway_peering_attachment_accepter" "dr" {
  provider = aws.dr
  transit_gateway_attachment_id = data.aws_ec2_transit_gateway_peering_attachment.dr.id
}

  2. 0

    When you create a Transit Gateway in terraform, the resource exposes two configuration options: default_route_table_propagation and default_route_table_association. These can be set to disable if you’d like to self-manage associations/propagations in your terraform configs.

    Additionally for traffic to flow across regions, you will have to setup a cross-region TGW peering with static routes configured in the route tables on both side.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search