skip to Main Content

Here’s my setup:

  • using AWS Organizations to segregate different application stacks; each Organizational Unit has separate AWS accounts to separate Dev and Prod.
  • one "Shared services" AWS account with a CodeArtifact repository containing private Python packages, including common packages for use by multiple OUs.

This works: I’ve been able to set up cross-org access to CodeArtifact and am able to pull down a package from CodeArtifact in Account Shared Services to say an EC2 instance in Account A.

I am setting up an AWS Glue job in Account A, and I want it to use some packages from my CodeArtifact repo.
This works: If I run the Glue job outside my VPC, no problem, I’m able to access CodeArtifact.

This doesn’t work: However, I want to run Glue within my VPC in Account A (to access other resources running in the VPC). It looks like when I do that, I lose the ability to connect to CodeArtifact.

This seems to be because when Glue runs within a VPC, AWS automatically creates an elastic network interface without any public IPs (described in more detail here). AWS suggests using a NAT gateway to get around this.

Now, if my CodeArtifact repo were also within Account A (where my Glue job is), I think I could create a VPC endpoint for it that I could use to connect from Glue from within the VPC. Is there some way to do something similar given its in a different account? Everything I’m talking about here is all within AWS, I don’t want to create a NAT gateway just to connect to an AWS service from another AWS service.

2

Answers


  1. Chosen as BEST ANSWER

    My question stemmed from an unclear understanding of the CodeArtifact service and VPC endpoints. Specifically: when a VPC endpoint is connected to an AWS service, it is simply a pathway for resources in a private subnet in the VPC to connect to the service without having access to the Internet. The service itself is global and is not account-specific.

    In other words, to solve for the scenario described in my question I was able to simply create a CodeArtifact VPC endpoint (I only needed the repository endpoint, not the api one) within Account A. This gave the Glue job running within the Account A VPC access to the global CodeArtifact service, and so long as I had the necessary permissions set for cross-account access on the CodeArtifact domain owned by the Account Shared Services account, I was able to download packages for use in my Glue job.


  2. Many AWS Services struggle with cross-account things. Drop AWS Support a ticket and they will tell you the details, it’s the only guaranteed way of getting a correct answer on the topic as they often don’t publish these kinds of nuances in any of the publicly available documentation.

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search