Amazon Web Services Question posted in
The official Amazon Web Services documentation can be found here.

Amazon web services – s3 touch not running from Lambda

I created bucket and folder name and I am passing those values in AWS LAMBDA .

First print is showing me both bucket name and folder name but I think s3.touch is not running .

Not able to find error . Exception is not showing anything . I am not sure if it is IAM issue.
How Can I display error I am getting and what is resolution of that ?

import boto3
import uuid

import s3fs

# S3 bucket info
s3 = s3fs.S3FileSystem(anon=False)

def lambda_handler(event, context):
    #below line is added only to make it run from local pycharm
    client = boto3.client('s3')
    input_archive_folder = 'test'     # event['inputArchiveFolder']
    output_path = input_archive_folder+'_'+str(uuid.uuid4()) + "/" + "output"
    file_row_limit = 600 # event['fileChunkSize'] ----No. of records we want to put in file
    file_delimiter =','   # event['fileDelimiter']

    record = event['Records'][0]
    print(record)
    bucket = record['s3']['bucket']['name']
    key = record['s3']['object']['key']
    print(key)
    create_start_indicator(bucket, output_path)

def create_start_indicator(bucket, folder_name):
    print(f'IN create_start_indicator {bucket}    {folder_name}')
    try:
        response = s3.touch(bucket + "/" + folder_name + "/_started")
        print(response)
    except error as e:
        print("Unexpected error: %s" % e)
Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    Thank you everyone for your time and effort. After reading document shared above which explains why we should not use s3FS . I have changed my code to work with s3 apis and it is working fine.


  2. 0

    Your exception handler is incorrect and is causing a NameError, which causes your print statement to be skipped. I would, however, expect AWS Lambda to make an appropriate log indicating this failure.

    Use except Exception as e rather than except error as e, for example:

    try:
    response = s3.touch(bucket + "/" + folder_name + "/_started")
    print("Response:", response)
except Exception as e:
    print("Unexpected error:", e)

    To correct the bigger issue, which is the undefined s3, use some variant of:

    import s3fs
s3 = s3fs.S3FileSystem(anon=False)
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search