In my Nodejs app, I have been using aws-sdk (2.x) to get S3 objects. Below code works pretty fine. const AWS = require("aws-sdk"); S3 = new AWS.S3({ accessKeyId: 'actual_accessKeyId', secretAccessKey: 'actual_secretAccessKey', region: 'region', signatureVersion: "v4", }); const listBucketContent = (filePath,…
I'm encountering an issue with the AWS SDK for Java v2 specifically for S3, but only on one particular Mac computer. The same code works fine on other machines (both Mac and Ubuntu). Interestingly, the AWS SDK v1 works perfectly…
I have duplicity running for years, stable and without problems. Now I encountered the following problem when I backup my files to Amazon S3. All files in the format "duplicity-full.YYYYMMDD1ThhmmssZ.volX.difftar.gpg" are written to Amazon S3 successfully. At least I can…
Lambda getting access denied when trying to copy the data in same bucket. Here is my Bucket Policy: { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": "*", "Action": [ "s3:GetObject", "s3:GetObjectTagging", "s3:PutObject", "s3:PutObjectTagging" ], "Resource": "arn:aws:s3:::vgi-esf-eng-us-east-1-narwhal-bfb-automated-input/*" }, { "Effect":…
In my Nextjs project I'm trying to copy files using (CopyObjectCommand). Using @aws-sdk/client-s3: 3.616.0 The PutObjectCommand and DeleteObjectCommand are working fine. However, the CopyObjectCommand returns an error Below is my code // Copy S3 Object export const copyS3Object = (sourceKey,…
When using a form POST to upload a file to an S3 bucket, S3 ignores the Content-Type/MIME type of the file. The file is uploaded successfully. The file form data field contains a Blob with a Content-Type of video/x-matroska;codecs=avc1,pcm. The…
I'm trying to read files on s3 using Flink 1.18.1. I'm moving plugin jar to /opt/flink/plugins/s3-fs-hadoop/ in my Dockerfile. I checked it's OK the plugin is in the right folder but when I'm running my pod I have: Caused by:…
export async function* initiateObjectStream( Key: string, start: number, end: number, ): AsyncGenerator<any, any, unknown> { const streamRange = `bytes=${start}-${end}`; const getObjectCommand = new GetObjectCommand({ Bucket: bucket, Key, Range: streamRange, }); const { Body: chunks} = await s3Client.send(getObjectCommand); for await (const…
I have an s3 route and I have to split it in bucket, prefix and key using JQ. "s3://bucket/folder1/folder2/folder3/key.txt" -> { bucket: bucket, prefix: folder1/folder2/folder3, key: key.txt }
We believe that the S3 checksum is useless for huge files larger than 1GB, as it is a further hashed value of chunks separated by an arbitrary number of bytes. There is a 1 GB file uploaded to AWS S3.…