Amazon Web Services Question posted in
The official Amazon Web Services documentation can be found here.

Amazon web services – How to ensure compatibility between my local dev environment and AWS Lambda runtime (AWS CDK V2)

I am currently having an issue with Python packages on AWS Lambda.

I have defined a Lambda layer like this:

        my_layer = _lambda.LayerVersion(
        self, "MyLayer",
        code=_lambda.Code.from_asset("layer_code_directory"), 
        compatible_runtimes=[_lambda.Runtime.PYTHON_3_12], 
        description="Lambda Layer for common dependancies"
    )

The layer is used for common dependencies that most of my Lambdas share. One of them is the cryptography package. The pip install all the packages in the requirements.txt into "layer_code_directory". The CDK (afaik) packages those into a zip and then uploads in to the lambda layer.

However, when I try to run my Lambda, I get an error, that some files from cryptography are missing:

Runtime.ImportModuleError: Unable to import module ‘MyAuthorizer’: /opt/python/cryptography/hazmat/bindings/_rust.abi3.so: cannot open shared object file: No such file or directory

I have debugged this by checking whether the package is available to the Lambda and what paths those files have that are missing. I crosschecked them with the error message and all the files are available in the correct places.

From some Google research I now assume, that the version of cryptography, I install locally via pip is not compatible with the lambda runtime, and afaik, the pip installs different files depending on what OS you are on.

On my first tries, I was on Alpine (Official Python docker container), and I have now moved to a container based on the AWS Python, Lambda, and Image. I assumed, when I bundle packages installed in this environment, the packages would be compatible with AWS Lambda. But I still got the same error message.

How to solve this issue or what else there could be?

I assume, that deploying Lambda code and packages via AWS CDK is a very common practice, but using Google, AWS, Docs, and LLMs.

I was unable to find the solution or a working example.

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    I have found a solution:

    pip install
    --platform manylinux2014_x86_64
    --target=package
    --implementation cp
    --python-version 3.x
    --only-binary=:all: --upgrade
    <package_name>

    This ensures all packages are compatible with the aws lambda runtime. It is also mentioned in this part of the documentation: https://docs.aws.amazon.com/lambda/latest/dg/python-package.html


  2. 0

    Things that seems to me as they could be the issue:

    First, im talking about @aws-cdk/aws-lambda-python-alpha module, just to make sure we are on the same CDK.

    Base on the CDK doc’s if theres requirements.txt in the entry dir, it will include the dependencies by itself, you don’t need to do it yourself if you use the CDK.
    Its also doing it for the lambda docker target, so it doesn’t matter what is your environment.

    If you didn’t changed anything in the source code it wouldn’t recreate it, hence its advisable to use lock file, so version are always the same and when the change is just a package version it will still rebundle.

    Not sure if relevant but might be:
    And use py 3.11 (base on lambda doc’s, couldn’t find a place say that 3.12 is supported, saw a place mention that 3.11 is the latest but might be outdated).

    Those are a bit obvious so if this is not the case, please share your file structure and I’ll try to update answer with fitting solution.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search