When the CLI is used to copy from one S3 path to another, it shows progress:
$ aws s3 cp s3://[source_path] s3://[destination_path]
Completed 264.0 MiB/1.6 GiB (40.4 MiB/s) with 1 file(s) remaining
How is it possible to get this information? The CopyObject
API call doesn’t seem to return any progress information, nor does UploadPartCopy
.
Large files can be copied part-wise with the client knowing how many parts have been copied so far, but the command-line output shown above is not showing the number of parts copied, as the typical part size is several hundred megabytes (up to 5GB), and the file I copied to get this sample output is not multi-part anyway.
2
Answers
Actually, the CLI does copy each part, all in parallel and displays the total size of all parts copied so far.
I thought the object was a single part or a few parts, but the CLI actually creates rather small parts by default - only 8MB - so the 1.6GB object had 193 parts.
You could debug it, will show http req & res, including size and transferred bytes.
aws s3 cp s3://source-destination local-destination --debug