amazon-s3 Questions

Amazon web services – How to generate url for the s3 object without expiring?

August 1, 2022
surya raj
2 Answers

I have uploaded an object with the client = boto3.client('s3', aws_access_key_id=AWS_ACCESS_KEY_ID, aws_secret_access_key=AWS_SECRET_ACCESS_KEY) response = client.put_object( Bucket=BUCKET_NAME, Body=in_mem_file.getvalue(), Key=str(img_name)) and I'm generating the URL by url = client.generate_presigned_url('get_object', Params={ 'Bucket': BUCKET_NAME, 'Key': str(img_name)}, ExpiresIn=518400) I need to generate the URL without…

VIEW QUESTION

Amazon web services – Data loss in AWS Lambda function when downloading file from S3 to /tmp

July 29, 2022
DD24
2 Answers

I have written a Lambda function in AWS to download a file from an S3 location to /tmp directory (local Lambda space). I am able to download the file however, the file size is changing here, not sure why? def…

VIEW QUESTION

Amazon web services – Cloudtrail: Insufficient permissions to access S3 bucket

July 28, 2022
Display name
2 Answers

I am trying to create a trail in Cloudtrail logging to S3 bucket but I am getting following error: Error: Error creating CloudTrail: InsufficientEncryptionPolicyException: Insufficient permissions to access S3 bucket $BUCKET_NAME or KMS key arn:aws:kms:eu-west-1:$ACCOUNT_ID:key/1234567890. Terraform is able to create…

VIEW QUESTION

Amazon web services – How to fix this error I’m getting from Amazon s3: ETagMissing: No access to ETag property on response. Check CORS configuration to expose ETag header

July 28, 2022
Marcus
2 Answers

I am running into a error "ETagMissing: No access to ETag property on response. Check CORS configuration to expose ETag header." when running a multipart upload to AWS. I searched the whole net and cant find a way to add…

VIEW QUESTION

Amazon web services – Strategy to store logs – Splunk

July 28, 2022
Sachin
2 Answers

Currently , we are collecting and sending our application logs through Kafka broker to Splunk for daily monitoring and it is around > 80GB/day. We are paying heavy amount to keep these logs into Splunk (90 Days retention period). We…

VIEW QUESTION

Amazon web services – AWS Glue error – Invalid input provided while running python shell program

July 27, 2022
ludwig
6 Answers

I have Glue job, a python shell code. When I try to run it I end up getting the below error. Job Name : xxxxx Job Run Id : yyyyyy failed to execute with exception Internal service error : Invalid…

VIEW QUESTION

Amazon web services – Error while generating AWS session token using java

July 27, 2022
John D
2 Answers

I am trying to use session token using AWS sts_client AWSSecurityTokenService sts_client = AWSSecurityTokenServiceClientBuilder.standard() .withEndpointConfiguration(new AwsClientBuilder.EndpointConfiguration("https://sts.ap-southeast-2.amazonaws.com", "ap-southeast-2")) .withRegion(Regions.AP_SOUTHEAST_2).build(); getting below error. java.lang.IllegalStateException: Only one of Region or EndpointConfiguration may be set. at com.amazonaws.client.builder.AwsClientBuilder.setRegion(AwsClientBuilder.java:450) ~[aws-java-sdk-core-1.12.234.jar:na] at com.amazonaws.client.builder.AwsClientBuilder.configureMutableProperties(AwsClientBuilder.java:424) ~[aws-java-sdk-core-1.12.234.jar:na] at com.amazonaws.client.builder.AwsSyncClientBuilder.build(AwsSyncClientBuilder.java:46) ~[aws-java-sdk-core-1.12.234.jar:na]…

VIEW QUESTION

Amazon web services – Not all folders returned by boto3 Bucket.objects.all()

July 22, 2022
Steven
3 Answers

My S3 bucket contains a bunch of files in a multilevel folder structure. I'm trying to identify the top level folders in the hierarchy, but objects.all() returns some but not all folders as distinct ObjectSummary objects. Why? Sample file structure:…

VIEW QUESTION

Amazon web services – S3 Static website give 403 forbidden access denied error when using urls

July 21, 2022
Ben
2 Answers

I have a static angular website which I have published to an S3 bucket. I enabled the Static website hosting on the S3 bucket, and also selected all files in the bucket and did the "Make public using ACL" option.…

VIEW QUESTION

Amazon web services – Extract specific column from csv stored in S3

July 20, 2022
D kashyap
3 Answers

I am querying Athena thru lambda. Results are getting stored in csv format in S3 bucket. The csv files has two columns - EventTime and instance id. I am reading csv file via one of function in my lambda handler:…

VIEW QUESTION