Question 71804 in Amazon Web Sevices Question posted in
The official Amazon Web Services documentation can be found here.

Move large volumes (> 50tb) of data from one s3 to another s3 in another account cost effectively – Amazon web services

I have some S3 buckets in one AWS account which have large amount of data (50+ Tbs)

I want it to move it to new S3 buckets in another account completely and use the 1st AWS account for another purpose.

The method I know is AWS CLI using s3 cp/s3 sync/s3 mv , but this would take days when running in my laptop
And I want it to be more cost effective when considering the data transfer also.

Buckets contain mainly zip files and rar files having size ranging from 1GB to 150+GB and also other files too.

Can someone suggest me methods to do this which would be cost effective as well as less time consuming .

Tags:

2

Answers


  1. 0

    If the destination bucket is in the same region as the source bucket (even if it’s in a different account), there’s no data transfer cost for running s3 cp/sync/mv according to the docs (check the Data transfer tab).

    For a fast solution, consider using S3 Transfer Acceleration, but note that this does incur transfer costs.

    Login or Signup to reply.
  2. 0

    You can use Skyplane which is much faster and cheaper than aws s3 cp (up to 110x for large files). Skyplane will automatically compress data to reduce egress costs, and will also give you cost estimates before running the transfer.

    You can transfer data between buckets in region A and region B with:

    skyplane cp -r s3://<region-A-bucket>/ s3://<region-B-bucket>/
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search