skip to Main Content

I’ve been dealing with this issue for the past few days now. I am trying to create a lambda layer with the following modules: cryptography, ldap3, and bonsai to use in my lambda_function that has a runtime of python3.11 and architecture type of x86_64. I have been using AWS CloudShell to create the lambda_layer zip file.

This is how I installed cryptography and ldap3:

-pip3.11 install     
    --platform manylinux2014_x86_64 
    --target=my-lambda-function 
    --implementation cp 
    --python-version 3.11.8 
    --only-binary=:all: --upgrade 
    cryptography

-pip3.11 install     
    --platform manylinux2014_x86_64 
    --target=my-lambda-function 
    --implementation cp 
    --python-version 3.11.8 
    --only-binary=:all: --upgrade 
    ldap3

These two modules work just fine I can call the module in my lambda function, the issue is with the bonsai module.
Unfortunately, I can’t install the bonsai module like this:

pip3.11 install     
    --platform manylinux2014_x86_64 
    --target=my-lambda-function 
    --implementation cp 
    --python-version 3.11.8 
    --only-binary=:all: --upgrade 
    bonsai

Because I get the following error when I run this command:

ERROR: Could not find a version that satisfies the requirement bonsai (from versions:none)
ERROR: No matching distribution found for bonsai

Therefore this is how I am installing bonsai:

pip3.11 install bonsai --target=my-lambda-function

But when I try to call the bonsai module in my lambda function I get this error:

Unable to import module 'lambda function': libldap_r-2.4.so.2 cannot open shared objectfile: No such file or directory

Does anybody know how to resolve this issue, I’ve completely hit a brick wall. Any help is welcome!

2

Answers


  1. You could try installing from the repo;
    pip install https://github.com/BonsaiAI/bonsai-python

    You could build in CodeBuild and push the artifact to s3.
    This article talks about using CloudFormation to create the layer with a ref to an s3 artifact;
    https://www.linkedin.com/pulse/lambda-layer-creation-cloudformation-martin-macecek/

    You could try to build inside a docker image with the same os as the lambda target os.

    That .so file isn’t available to the lambda os. I think that’s the problem to work around somehow.

    Login or Signup to reply.
  2. It’s much easier to build a lambda layer by using the Serverless Framework. I’ve built the layer for you. You can download it through this google drive link. This layer is for the x86_64 python3.11 runtime. Remember that you need to set the environment variable LD_LIBRARY_PATH to /opt/lib before using this layer.

    Test code:

    import json
    import cryptography
    import ldap3
    import bonsai
    
    def lambda_handler(event, context):
        # TODO implement
        print(cryptography.__version__)
        print(ldap3.__version__)
        print(bonsai.__version__)
        return {
            'statusCode': 200,
            'body': json.dumps('Hello from Lambda!')
        }
    
    

    Test result:

    Test Event Name
    test
    
    Response
    {
      "statusCode": 200,
      "body": ""Hello from Lambda!""
    }
    
    Function Logs
    START RequestId: 3cfbef06-dd11-4fba-bec8-48004ad2e959 Version: $LATEST
    42.0.5
    2.9.1
    1.5.2
    END RequestId: 3cfbef06-dd11-4fba-bec8-48004ad2e959
    REPORT RequestId: 3cfbef06-dd11-4fba-bec8-48004ad2e959  Duration: 8.01 ms   Billed Duration: 9 ms   Memory Size: 128 MB Max Memory Used: 67 MB  Init Duration: 506.47 ms
    
    Request ID
    3cfbef06-dd11-4fba-bec8-48004ad2e959
    

    If you don’t trust my code, you can follow the steps below to build it yourself.

    Step 1: Add a requirements.txt

    cryptography==42.0.5
    ldap3==2.9.1
    bonsai==1.5.2
    

    Step 2: Add a Dockerfile

    FROM public.ecr.aws/sam/build-python3.11:latest-x86_64
    RUN yum -y install openldap-devel
    

    Step 3: Modify serverless.yml

    service: example
    
    plugins:
      - serverless-python-requirements
    
    custom:
      pythonRequirements:
        dockerFile: Dockerfile
        dockerizePip: true
        layer: true
        dockerExtraFiles:
          - /lib64/libldap-2.4.so.2
          - /lib64/liblber-2.4.so.2
          - /lib64/libsasl2.so.3
          - /lib64/libssl3.so
          - /lib64/libsmime3.so
          - /lib64/libnss3.so
          - /lib64/libnssutil3.so
    
    provider:
      name: aws
      runtime: python3.11
      stage: dev
      region: us-east-1
      profile: dummy
    
    functions:
      dummy:
        handler: dummy.handler
    

    Step 4: Build the layer

    Simply do sls package -p pkg, then the layer zip named pythonRequirements.zip will be in pkg directory.

    Step 5: Move .so files to a separate directory

    Due to a bug in serverless-python-requirements, you need to unzip the pythonRequirements.zip, create a lib directory, move .so files into the new dir, and re-zip it.

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search