Amazon Web Services Question posted in
The official Amazon Web Services documentation can be found here.

Amazon web services – Importing the OpenAI Python dependency into AWS Lambda fails

I am trying to add the OpenAI Python library to my AWS Lambda function. I tried to add them via AWS Lambda Layers as described in this guide. However, when executing my code, the addition of this line:

import openai

leads to this error-response:

Response
{
  "errorMessage": "Unable to import module 'lambda_function': No module named 'pydantic_core._pydantic_core'",
  "errorType": "Runtime.ImportModuleError",
  "requestId": "c19b73e3-8c6d-4564-8be6-af7b03a79e00",
  "stackTrace": []
}

Function Logs
START RequestId: c19b73e3-8c6d-4564-8be6-af7b03a79e00 Version: $LATEST
[ERROR] Runtime.ImportModuleError: Unable to import module 'lambda_function': No module named 'pydantic_core._pydantic_core'
Traceback (most recent call last):END RequestId: c19b73e3-8c6d-4564-8be6-af7b03a79e00
REPORT RequestId: c19b73e3-8c6d-4564-8be6-af7b03a79e00  Duration: 5.02 ms   Billed Duration: 6 ms   Memory Size: 128 MB Max Memory Used: 40 MB  Init Duration: 166.39 ms

When following along the guide using the same package as them (requests), importing that package (requests) doesn’t lead to an error.

Are there possible explanations for this behavior? I don’t know where to start looking for fixes. Or are there any alternative methods of importing the library?

Tags:

2

Answers


  1. 0

    Your uploaded Lambda environment / Layer does not have pydantic installed which is a requirement of the openai python package.

    Ensure that all the required packages are installed in your folder where you have your python Lambda code before zipping and uploading to AWS via the console.

    That being said, there is a similar question and answer that refers to the same part of the documentation like you.

    Login or Signup to reply.
  2. 0

    I was running into the same issue over and over again, and I finally found the solution. You have a couple of options to tackle this, in my case, I used Serverless Framework but if you’re even using SAM that’s fine as well.

    i) So the quick approach is to dockerize your lambda, and change the architecture as arm instead of x86. You can dockerize your lambda using this Dockerfile

    FROM public.ecr.aws/lambda/python:3.10
COPY requirements.txt .
RUN  pip3 install -r requirements.txt --target "${LAMBDA_TASK_ROOT}"
COPY . ${LAMBDA_TASK_ROOT}
CMD ["handler.generate_code"]

    ii) The second option is, to use the serverless-python-requrements plugin if you’re using the serverless framework and have the followign configuration

     pythonRequirements:
    dockerizePip: non-linux
    dockerRunCmdExtraArgs: ["--platform", "linux/x86_64"]
    usePipenv: false
    slim: true
    layer: true

    This one is flexible because you can use either x86 or arm depending on the dockerRunCmdExtraArgs
    Hope this helps.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search