Question 245753 in Amazon Web Sevices Question posted in
The official Amazon Web Services documentation can be found here.

Amazon web services – How to move files from a search from one bucket to another?

I have a bucket in S3 that has a folder with several objects that contain the date in the name of each file and I want to move only the objects from a specific date to another folder in the same bucket, but it is impractical to do this manually as there are more than 6k files.
I did a search to return only the objects I wanted but when I try to move or copy the objects it doesn’t work

I did this command to return the objects I wanted and I tried this to move and copy

aws s3 ls s3://bucket-name | grep "20230904" | grep "cdrquoted"
aws s3 mv s3://bucket-name | grep "20230904" | grep "cdrquoted" s3://my-bucket/
Tags:

2

Answers


  1. 0

    You can use S3 sync command with –include option.
    Docs: https://docs.aws.amazon.com/cli/latest/reference/s3/#use-of-exclude-and-include-filters
    In this case your command will look like this:

    aws s3 sync s3://from s3://to --include /20230904/cdrquoted/*
    Login or Signup to reply.
  2. 0

    Many of the S3 sub-commands in the AWS CLI support exclude and include filters, including mv. You can use a combination of these filters to specify which objects to operate on.

    In your case, since you only want to include objects with a string, you need to start by inverting the normal logic, so you can start with --exclude="*", which will exclude all objects. Then --include="*20230904*" will include all objects that include the target string.

    Putting it together looks like:

    aws --dryrun s3 mv s3://bucket-name/ s3://my-bucket/ --recursive --exclude="*" --include="*20230904*"

    Note the use of quotes to prevent any shell wildcard expansion from happening on Unix-like systems. Also, the inclusion of the --dryrun option to prevent the command from doing work, just showing what it would do. This lets you verify the command will operate correctly before making changes to the bucket.

    After you’ve verified the command does what you’d like, remove the --dryrun flag and run the command to perform the actual move.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search