I am using the terminal to copy a file from an s3 bucket to my local machine but I keep getting the error:
fatal error: An error occurred (404) when calling the HeadObject operation: Key "file_000" does not exist
I am using the command:
usr/local/bin/aws s3 cp s3://{bucket}/file_000 /Users/user/Documents/Docs/dir/new_file.csv
I know the file exists using:
aws s3 ls s3://{bucket} --recursive --human-readable
and shows up as:
2022-08-04 15:53:12 21.2 MiB file_000
I have tried adding --recursive
to the end of the command. The command goes through but then creates an empty directory named new_file.csv
Is there anything I can do to solve this?
3
Answers
I ran into this same problem, what worked for me was going back to the s3 bucket and confirming the name of the object in the bucket is the same as inside of my code. I realized that it wasn’t and once I corrected that and re-ran the program everything worked fine.
For anyone, who will ran into this problem lately, it could happen, when you are trying to copy files from folder.
Imagine, I have a large folder with thousand of files, and I want to copy just small amount of file to another folder. In that case if I have s3 path
s3://object1/object2/
whereobject2
is folder, I need to add--recursive
flag to mine request.And the whole CLI command will be:
P.S. This is not the only one solution for the "Key ” does not exist" problem, and you might want to debug it further: https://repost.aws/knowledge-center/404-error-nosuchkey-s3
I came across this issue when I was trying to access one file from s3, it turns out I was requesting a file that does not exist! Hope it helps