Question 108218 in Azure Question posted in
The official documentation to get you started can be found here.

Error on copying AWS s3 bucket to Azure Storage using AZCopy

I have been trying to copy a AWS S3 bucket to my Azure Storage container following the Microsoft document Amazon S3 to Azure Storage by using AzCopy

But the operation fails with error as below

failed to perform copy command due to error: cannot start job due to error: cannot list objects, Access Denied.

I have tried to copy by authentication through both Azure AD and also SAS token with Full Access privileges (racwdl) but get same issue.

I have verified that the AWS keys set are of admin access and also verified that both my AWS bucket and Azure Storage Containers are public.

What else could be wrong? Any inputs would be helpful.

Also sharing the terminal outputs on using AD authorization and SAS token authorization.

On AD Authorization:

INFO: Scanning… INFO: Authenticating to destination using Azure AD
INFO: Instead of transferring from the ‘s3.amazonaws.com’ URL, in this
version of AzCopy we recommend you use a region-specific endpoint to
transfer from one specific region. E.g. s3.us-east-1.amazonaws.com or
a virtual-hosted reference to a single bucket. INFO: Any empty folders
will not be processed, because source and/or destination doesn’t have
full folder support

failed to perform copy command due to error: cannot start job due to
error: cannot list objects, Access Denied.

Using SAS Token Authorization

INFO: Scanning… INFO: Instead of transferring from the
‘s3.amazonaws.com’ URL, in this version of AzCopy we recommend you use
a region-specific endpoint to transfer from one specific region. E.g.
s3.us-east-1.amazonaws.com or a virtual-hosted reference to a single
bucket. INFO: Failed to create one or more destination container(s).
Your transfers may still succeed if the container already exists.
INFO: Any empty folders will not be processed, because source and/or
destination doesn’t have full folder support

failed to perform copy command due to error: cannot start job due to
error: cannot list objects, Access Denied.

Tags:

3

Answers


  1. Chosen as BEST ANSWER
    0

    I have managed to resolve this issue. Apparently the solution is to add the AWS credentials as environment variables as setting them on CLI as mentioned in the documents doesn't work.


  2. 0

    As stated above, the powershell script "set AWS_ACCESS_KEY_ID=….." doesn’t work as it doesn’t add the system environment variables in windows, so the access is denied to S3. To add the environment variables in Windows do the following:

    1. Press Windows Key + R then open "control sysdm.cpl"
    2. In System Properties click on Advanced -> Environment Variables
    3. Under System Variables add the variable name: AWS_ACCESS_KEY_ID and place your access key under the value.
    4. Add another system variable "AWS_SECRET_ACCESS_KEY" and place in the AWS secret key.
    5. Go back to powershell and you should be able to run azcopy and the permissions should work.
    6. If the permissions still do not work, then it might be an issue with the permissions in AWS.
    Login or Signup to reply.
  3. 0

    If you are using PowerShell to run the command, environment variables can be set as follows (without going into system variables)

    $env:AWS_ACCESS_KEY_ID="xxxxxxxxxxxx"
$env:AWS_SECRET_ACCESS_KEY="yyyyyyyyyyyy"

    To list the environment variables in PowerShell, use

    dir env:
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search