Question 254811 in Amazon Web Sevices Question posted in
The official Amazon Web Services documentation can be found here.

Amazon web services – How can I copy one s3 bucket between accounts

I have two sets of aws credentials and two buckets s3://foo and s3://bar how can I recursively move the contents of foo to bar?

I have tried rclone but this downloads the files first before uploading.

And aws s3 copy only works with one set of aws credentials.

Any other ideas?

Tags:

2

Answers


  1. 0

    You could use Amazon S3 Batch Replication if you are Ok enabling versioning on the buckets.

    Login or Signup to reply.
  2. 0

    This solution uses Python Boto3 in a script to recursively copy objects from one S3 bucket to another in a different account.

    1. You may need to create an IAM User or Role in the destination account that allows the account to get objects from the source S3 bucket and put objects to the destination S3 bucket.
    "Statement": [
    {
        "Effect": "Allow",
        "Action": [
            "s3:GetObject*",
            "s3:PutObject*"
        ],
        "Resource": [
            "arn:aws:s3:::SOURCE_BUCKET/*",
            "arn:aws:s3:::DESTINATION_BUCKET/*"
        ]
    }
]
    1. Create separate Boto3 S3 resources using credentials in the source and destination accounts.
    source_s3 = boto3.Session(profile_name="source").resource("s3")
dest_s3 = boto3.Session(profile_name="destination").resource("s3")
    1. Iteratively copy objects over using the source and destination Boto3 S3 resources.
    for obj in source_s3.Bucket("SOURCE_BUCKET_NAME").objects.all():
    source_key = obj.key

    kwargs = {
        "Bucket": "DEST_BUCKET_NAME",
        "Key": destination_key,
        "CopySource": {"Bucket": "SOURCE_BUCKET_NAME", "Key": source_key},
    }
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search