Can Terraform whitelist a cross-account source security group ID on AWS?

I have a central AWS account that I am running some AMI builds in with Packer. I have a secondary account where I also need to run builds in on occasion but I can only afford to keep my CI/CD servers running in one account. That being said, I am trying to create a restrictive security group to only allow inbound SSH access to the cross-account temporary packer instance from the security group of my CI/CD server in the central account.

I have tried using the cross-account ID as a prefix in the source_security_group_id argument in the aws_security_group_rule resource block like so: source_security_group_id = "12345678910:sg-0a32d4ae223c46901"

The plan runs fine, however, terraform apply fails with the following error:

 "Error: authorizing Security Group (sg-00ff5f609744440ad) 
Rule (sgrule-3017511228345): InvalidGroupId.Malformed: 
Invalid id: "12345678910:sg-0a32d4ae223c46901" 
(expecting "sg-...") status code: 400, request id: xyz"

When I pass in the raw SG id without the account prefix, it also fails with a Security group not found error as TF obviously only knows to look for said SG in the secondary account, not the central account.

Code:

resource "aws_security_group" "builder_sg" {
  name        = "Builder-Dev-SG"
  description = "SG for dev account builds"
  vpc_id      = data.aws_vpc.staging_vpc.id
}

resource "aws_security_group_rule" "builder_sg_ingress_0" {
  security_group_id = aws_security_group.builder_sg.id
  description       = "Allow SSH from central account"

  type      = "ingress"
  protocol  = "tcp"
  from_port = 22
  to_port   = 22

  # Only allow SSH from cross-account CI/CD SG
  # Format: <cross-account-id:sg-id>
  source_security_group_id = "12345678910:sg-0a32d4ae223c46901"
}

Is there another way to accomplish this in a simple fashion?

TF_VERSION: 1.3.7