Amazon Web Services Question posted in
The official Amazon Web Services documentation can be found here.

Amazon web services – Unable to load modules using AWS Lambda

I’m trying to run a AWS Lambda function with custom modules (e.g. pyodbc) so I’m using a Layer to include them via a zip file. I was following these tutorials:

https://docs.aws.amazon.com/lambda/latest/dg/python-layers.html
https://docs.aws.amazon.com/lambda/latest/dg/packaging-layers.html

However, I keep getting the error message when testing my function:

Response:
{
  "errorMessage": "Unable to import module 'lambda_function': No module named 'pyodbc'",
  "errorType": "Runtime.ImportModuleError",
  "requestId": "",
  "stackTrace": []
}

I’ve tried the following:

  1. Creating a deployment package through a project directory using ‘pip install –target ./package pyodbc’, zipping it, and uploading to Layers. This leads to the following directory structure within the zip:
./
  pyodbc-5.2.0.dist-info/

which didn’t work, so I tried next.

  1. Creating a deployment package using a virtual environment using ‘pip install –target pyodbc’ WITHIN the environment. This leads to the following directory structure within the zip:
python/
  Lib/
    site-packages/
      pyodbc-5.2.0.dist-info/

This didn’t work. I saw other threads where it seems an additional python subfolder before ‘site-packages’, so I also tested the following directory structure and uploaded the zip:

python/
  Lib/
    python3.11/
      site-packages/
        pyodbc-5.2.0.dist-info/

I also reviewed these articles in StackExchange, but the solutions haven’t worked for me (sorry I couldn’t post full links as my question was getting flagged as spam)

62404733/unable-to-import-modules-from-aws-lambda-layer

75419822/unable-to-import-module-aws-lambda-function

67062313/cant-import-packages-from-layers-in-aws-lambda

76105233/import-module-not-working-in-aws-lambda-when-using-a-layer-cannot-find-packag

67118576/aws-lambda-unable-to-import-module-from-layer

55695187/import-libraries-in-lambda-layers

I’d appreciate any help or comments. Sorry if this may seem trivial, I’m kinda new to Lambda Layers.

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    Solution:

    I forgot to specify, but besides the lambda_function.py I was also using some helper functions to keep my code ordered. However, it seems that the modules for the AWS Lambda Layers are not visible for these, so ideally all the imports should be in the main lambda_function.py that is invoked by the handler.

    I didn't find any information about this on the documentation, so I'm posting the solution in case anyone finds it useful.

    1. When using AWS Lambda Layers, all module imports should be on the main function called by the handler (e.g. lambda_function.py)

    2. Python version for packages should be the same as the AWS Lambda runtime environment (follow this tutorial https://docs.aws.amazon.com/lambda/latest/dg/python-layers.html)

    3. Changes take some time to reflect (even if AWS says it's ready), so after deploying and uploading layer, give it 2-3 minutes for the backend to update (only in case it doesn't work initially)

    Edit: also want to add that, for the specific case of pyodbc, you also need to install driver libraries (which don't come with the pip install). Here are a couple of guides on how to set it up:

    https://medium.com/@narayan.anurag/breaking-the-ice-between-aws-lambda-pyodbc-6f53d5e2bd26

    https://randywestergren.com/building-pyodbc-for-lambdas-python-3-9-runtime/


  2. 0

    The zipped structure should look like this:

    python/
  lib/
    python3.11/
      site-packages/
        pyodbc/
        pyodbc-5.2.0.dist-info/

    Did you verify that the pyodbc library is available in the Layer by extracting the Layer zip file and checking the structure?

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search