Question 355747 in Amazon Web Sevices Question posted in
The official Amazon Web Services documentation can be found here.

Amazon web services – Verify that boto3 in EC2 is using VPC endpoint to connect to S3?

How can I verify that boto3.client('s3') in EC2 is using VPC endpoint?

In AWS, I have a created VPC Endpoint (service=s3, type=gateway). From what I understand, boto automagically selects its client(endpoint_url). I am ingesting terabytes of data, so I don’t want to get charged egress fees when writing from EC2 to S3.

Desktop

>>> import boto3
>>> conn = boto3.client('s3')
>>> conn._endpoint

"s3(https://s3.amazonaws.com)"

AWS Batch Job – Fargate within VPC

I want more information here. Is it pointed to my VPC Endpoint ID/ARN?

>>> import boto3
>>> conn = boto3.client('s3')
>>> conn._endpoint

"s3(https://s3.amazonaws.com)"

Here, boto3 acknowledges the existence of my VPC Endpoint

>>> from pprint import pprint
>>> import boto3

>>> conn = boto3.client('ec2')    #<-- note 'ec2' not 's3'
>>> resp = conn.c.describe_vpc_endpoints()
>>> pprint(resp['VpcEndpoints'])
Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    S3 Logs

    Another indirect solution without use of boto

    Using S3 logs, I am able to see the IP addresses of requests made to S3. The IP matches the 172.31.0.0/16 CIDR of my VPC as opposed to a public IP, so I conclude that my endpoint/gateway is working.

    Process

    • Created a new bucket to store logs
    • In my original bucket that I want to monitor: Properties > Server Access Logging
      • Enable
      • Point to newly created log bucket
    • Wait 45 min for logs to appear

    Supplemental Info


  2. 0

    The easiest way I can think of is to use bucket policies. You can restrict bucket access to a specific endpoint. If your requests still work after the bucket policy is applied, then it means your requests are going through the VPC endpoint.

    Example: Restrict access to a specific endpoint

    {
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Allow-access-to-specific-VPCE",
      "Effect": "Deny",
      "Principal": "*",
      "Action": ["s3:PutObject", "s3:GetObject", "s3:DeleteObject"],
      "Resource": ["arn:aws:s3:::bucket_name",
                   "arn:aws:s3:::bucket_name/*"],
      "Condition": {
        "StringNotEquals": {
          "aws:sourceVpce": "vpce-1a2b3c4d"
        }
      }
    }
  ]
}
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search